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— > The  Military  Message  Experiment  (MME)  is  designed  to  evaluate  the  utility  of  user-oriented 
message  processing  systems  in  a military  environment  and  to  aid  in  determining  the  features  useful 
in  such  a system.  The  experiment  is  a cooperative  effort  between  the  Commander-In-Chief,  Pacific, 
the  Navy,  and  the  Defense  Advanced  Research  Projects  Agency.  To  conduct  the  experiment,  a 
PDP-10-based  system  has  been  installed  at  CINCPAC  Headquarters  for  use  by  a portion  of  the 
Operations  Directorate.  The  message  processing  functionality  is  provided  by  SIGMA,  a program 
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20.  Abstract  (Continued) 

written  by  the  Information'  Sciences  Institute  of  the  University  of  Southern  California.  It  is  supported! 
by  the  TENEX  operating  system,  and  the  user  terminals  are  modified  HP-2649A  CRTs.  The  Staff 
began  limited  experimental  use  of  the  system  in  July  1978.  This  interim  report  discusses  the 
impressions  gained  during  this  period;  two  additional  reports  are  planned,  the  final  to  be  published 
in  December  1979. 

Hie  MME  system  is  designed  to  give  the  user  the  capability  to  handle  his  message  traffic  (both 
incoming  and  outgoing,  formal  and  informal)  on  the  system.  'Hie  system  enforces  multilevel  security 
rules  based  on  a modification  of  the  security  kernel  model  developed  at  Mitre.  The  rule  enforcement 
is  not  rigorous  enough  for  certification,  but  it  is  sufficiently  rigorous  to  determine  the  effects  on  the 
users’  interactions  with  the  system.  Most  of  the  functions  needed  for  a user’s  message-related  tasks 
are  provided  (or  will  be  provided  before  the  end  of  the  experiment)  by  the  system:  message  dis- 
tribution and  redistribution,  “electronic  readboard”  construction,  message  filing,  message  replies, 
message  commenting  and  “chopping”,  and  message  release.  ^ 

The  system  has  not  yet  been  in  use  by  the  staff  members  as  an  integral  part  of  their  day-to-day 
activity  long  enough  to  justify  any  conclusions.  The  general  observation  is  that  the  system  is  per- 
ceived by  the  users  as  useful  and  that  the  use  of  the  system  will  increase  as  the  reliability  and 
functionality  of  the  system  increase.  As  the  users  transition  to  the  automated  system,  the  data 
gathering  and  analyses  should  provide  information  that  will  be  useful  in  assessing  the  utility  of  the 
systems  and  in  determining  the  appropriate  set  of  functions  needed  in  a user-oriented  military 
message  processing  system. 
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MILITARY  MESSAGE  EXPERIMENT 
QUICK  LOOK  REPORT 

SECTION  1 
EXECUTIVE  SUMMARY 


This  report  is  the  first  of  three  reports  (two  quick-look  and  one  final) 
to  be  prepared  during  the  Military  Message  Experiment  (MME);  this  first  report 
covers  the  period  from  May  1977  to  Nov  1978.  It  reports  the  objectives  of  the 
experiment,  the  specific  system  being  used  by  the  staff  of  the  Commander-in- 
Chief  Pacific  (CINCPAC)  to  perform  the  experiment,  problems  and  deficiencies 
encountered  to  date,  general  plans  for  both  future  experimental  efforts  and 
system  modifications,  and  observations  of  the  experiment  progress  to  date.  No 
conclusions  are  drawn  at  this  point  because  the  report  period  consisted 
primarily  of  experiment  set  up  and  Limited  Experimental  Use  (LEU)  by  the 
CINCPAC  Operations  Directorate  (J3). 

The  Military  Message  Experiment  is  an  effort  to  evaluate  in  a formal  manner 
the  utility  of  computer-aided  message  handling  in  a military  environment.  It 
is  also  designed  to  aid  in  the  determination  of  the  type  of  automation  needed 
in  message  processing  within  military  environments  such  as  that  at  CINCPAC. 

The  system  is  based  on  the  message  processing  systems  developed  by  the  Defense 
Advanced  Research  Projects  Agency  (DARPA)  that  are  used  to  pass  informal 
messages  over  the  ARPANET  and  on  the  additional  user  requirements  within  a 
military  staff.  Features  of  the  message  service  system  being  evaluated 
include  the  ability  to  receive,  route,  file,  draft,  edit,  "chop",  and  release 
formal  military  messages  and  internal  memos  at  all  levels  of  classification. 

In  order  to  establish  a framework  for  the  experiment,  a formal  Memorandum 
of  Agreement  (MOA)  f reference  (a)]  was  signed  between  the  Director  of  DARPA; 
Commander,  Naval  Telecommunications  Command;  Commander,  Naval  Electronic 
Systems  Command;  and  Commander-in-Chief,  Pacific,  in  December  1975  and  revised 
in  September  1978.  Under  the  MOA,  DARPA  has  general  responsibility  for  the 
development  of  the  MME  system;  COMNAVTELCOM  acts  as  the  single  point  of 
contact  for  the  Navy;  COMNAVELEXSYSCOM  has  general  responsibility  for 
evaluation  of  the  MME;  and  CINCPAC  has  general  responsibility  for  providing 
facilities,  services,  personnel,  and  support  functions. 

The  MOA  identifies  the  following  specific  objectives: 

(a)  determine  and  demonstrate  the  usefulness  of  automated  message 
capabilities  and  the  necessary  features  to  support  a military  message 
handling  system  in  an  operational  environment; 

(b)  determine  the  effect  of  an  automated  message  handling  system  on 
operational  procedures,  manpower,  and  logistics  in  an  operational 
environment ; 

(c)  determine  the  training  requirements  associated  with  automated  message 
handling  systems; 
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(d)  determine  the  characteristics  of  an  acceptable  user  interface  for  an 
interactive  automated  message  handling  system; 

(e)  determine  multi-level  security  design  characteristics  and  their 
impact  on  the  user  interface; 

(f)  obtain  the  data  necessary  to  assist  in  the  future  design  and 
development  of  a family  of  automated  message  handling  systems  for  DoD 
use. 

The  experiment  consists  of  (a)  the  installation  at  CINCPAC  Headquarters, 
Camp  Smith,  Hawaii,  of  a fully  automated  message  handling  system  linked 
directly  to  the  military  AUTODIN  communications  system  via  the  Local  Digital 
Message  Exchange  (LDMX)  and  providing  service  to  selected  divisions  within  the 
Operations  Directorate  (J3);  (b)  on-site  operational  and  maintenance  support; 
(c)  on-site  training  and  training  facilities;  and  (d)  an  evaluation  team  made 
up  of  users,  on-site  technical  specialists,  and  personnel  from  NHL,  NAVSEA, 
NAVELEX,  MITRE,  and  CTEC.  Information  is  obtained  from  machine-compiled  user 
statistics,  user  surveys,  controlled  exercises,  and  the  monitoring  of  staff 
exercises.  The  determination  of  military  utility,  however,  ultimately  will 
depend  on  a subjective  judgment  of  the  value  of  improved  message  handling 
efficiency  versus  expense  and  whatever  other  shortcomings  are  discovered 
during  the  experiment. 

The  system  installed  at  CINCPAC  for  the  experiment  consists  of  a message 
processing  computer  program,  SIGMA,  and  an  operating  system,  TENEX,  resident 
in  a Digital  Equipment  Corporation  (DEC)  PDP-10  computer  with  one  million 
(1,048,576)  words  of  memory  and  a XL  central  processing  unit  (CPU).  The  user 
terminals  are  commercial  Hewlett-Packard  HP-2649A  CRTs.  They  have  additional 
firmware  written  by  ISI  and  have  been  modified  to  meet  the  TEMPEST 
requirements  for  handling  classified  information.  The  computer  and  most  of 
the  terminals  are  located  within  the  CINCPAC  security-controlled  blockhouse 
and  operate  "system  high";  external  remote  terminals  are  interfaced  via 
encrypted  lines.  Because  of  the  limited  number  of  terminals  to  be  used  in  the 
experiment,  it  was  decided  to  concentrate  them  within  a selected  number  of  J3 
divisions  rather  than  spreading  them  thinly  among  all  the  divisions.  At  the 
end  of  November,  1978,  there  were  16  terminals  installed  in  the  J3  spaces. 
Current  plans  are  to  install  additional  terminals  as  available  (a  minimum  of 
25  total  terminals). 

The  experiment  started  at  CINCPAC  in  May  1977  with  the  installation  of  the 
system  and  the  start  of  a prolonged  shakedown  and  training  period.  The  first 
version  of  SIGMA  did  not  provide  adequate  responsiveness  to  the  users.  While 
some  of  the  problems  in  responsiveness  were  caused  by  inefficiencies  in  the 
software  and  were  expected,  it  appears  that  the  user  services  provided  by 
SIGMA  require  more  processor  power  than  the  developers  and  evaluators 
originally  estimated.  The  installation  of  SIGMA  Resease  2.0,  in  June  1978, 
and  general  improvements  to  the  hardware,  led  to  a period  of  Limited 
Experimental  Use  (LEU)  by  the  J3  staff  commencing  in  July.  The  system  at  that 
time  was  able  to  support  concurrently  up  to  10-15  users  with  slow,  but 
generally  adequate  response,  under  light  loading.  SIGMA  Release  2.1  in 
September  1978  further  upgraded  the  responsiveness.  In  October,  1978,  the 
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original  KA  Central  Processing  Unit  was  replaced  with  the  more  powerful  KL, 
and  the  final  memory  increment  was  installed  for  an  increase  from  the  original 
256K  to  the  present  one-million  word  capacity.  This  upgrade  has  provided  a 
more  acceptable  response  time  for  the  users  and  allowed  for  the  expansion  of 
the  system.  The  system  now  provides  a basis  for  a realistic  experiment,  and 
no  further  major  upgrade  of  the  hardware  is  planned. 

Three  additional  software  releases  (upgrades)  are  planned: 

SIGMA  2.2,  January  1979,  will  provide  the  improved  capabilities  needed  for 
the  coordinate/release  function  and  increased  responsiveness. 

SIGMA  2.3,  April  1979,  will  provide  discretionary  access  controls  and  some 
additional  user  features. 

SIGMA  2.4,  July  1979,  is  the  last  planned  upgrade  and  is  intended  to 
provide  improved  file  archiving  capabilities  and  some  additional  user 
features . 

Requests  for  additional  changes  will  be  carefully  reviewed  to  ensure  that 
a sufficiently  stable  baseline  is  maintained  in  order  to  derive  reliable 
experiment  conclusions.  During  the  final  evaluation  of  the  experiment,  the 
impact  of  changing  experimental  parameters  on  the  result  will  be  investigated. 
In  addition,  user-requested  changes  provide  an  important  input  during  the 
evaluation  of  the  experiment. 

Of  considerable  importance  to  the  development  of  military  message  systems 
is  the  handling  of  messages  and  information  with  different  security 
classifications  (the  multi-level  security  problem)  and  messages  with  various 
restrictions  on  the  distribution  (discretionary  access  controls  that  depend  on 
a particular  command's  policy).  The  security  policy  is  intended  to  ensure 
that  only  those  with  both  the  requisite  security  clearance  and  a bonafide 
"need-to-know"  can  obtain  access  to  information  contained  within  the  system. 
The  emerging  security-kernel  technology,  based  in  general  on  the  model 
described  by  Bell  and  Lapadula  in  reference  (b),  was  chosen  as  the  basis  for 
enforcement  of  the  security  controls  for  the  MME  system.  See  references 
(c)-(e)  for  a discussion  of  the  security  considerations.  During  the 
development  of  the  MME  system,  it  was  decided  that  implementing  a security 
kernel  using  the  existing  hardware  and  software  would  result  in  a high  cost 
and  a low  probability  of  successful  certification.  The  security  controls, 
therefore,  are  not  enforced  with  the  rigor  necessary  for  certification  for 
operation  in  an  "open"  security  environment.  (The  computer  system  and  all  the 
terminals  are  in  a secure  environment  for  the  experiment.)  The  approach  of 
designing  the  security  controls  prior  to  designing  the  system  software  and 
user  interface,  however,  has  resulted  in  identifying  and  clarifying  many  of 
the  important  issues  that  must  be  resolved  in  future  secure  message  processing 
systems.  There  are  no  other  large-scale  military  message  processing 
systems  having  user  interfaces  that  reflect  the  restrictions  imposed  by  strict 
adherence  to  a formal  security  policy.  The  preliminary  results  of  the  MME 
indicate  that  a facile  interface  can  be  provided  in  a security  kernel-based 
system  provided  certain  minor  relaxations  in  the  security  model  described  in 
reference  (b)  are  made.  The  security  issues  will  be  discussed  in  depth  in  the 
final  report. 
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The  MME  system  provides  users  the  capability  to  handle  all  message  and 
memo  processing  functions  directly  from  a CRT  (Cathode-Ray  Tube)  terminal  with 
ancillary  printers  to  provide  paper  copy  where  required.  The  experiment  does 
not  include  a precept  that  paper  processing  of  messages  is  bad.  In  fact,  one 
of  the  experiment  goals  is  to  determine  the  proper  balance  between  hard  and 
soft  information,  i.e.,  between  the  use  of  paper  copies  and  displays. 

The  terminal  screen  itself  is  divided  into  different  areas  (or  windows)  so 
that  more  than  one  message  can  be  viewed  simultaneously.  A set  of  security 
lights  keeps  the  user  informed  of  the  security  level  of  the  information  in 
each  window  and  of  the  security  level  of  the  information  that  the  user  may  be 
typing  into  the  window;  another  set  of  security  lights  indicates  the  highest 
level  of  information  in  any  of  the  windows. 

Incoming  messages  from  the  LDMX  are  stored  in  a central  data  base  in  the 
MME.  Citations  (brief  identifying  summaries)  are  routed  directly  to  the 
administrative  section  of  J3  (J301)  who,  in  turn,  electronically  routes  them 
for  action  to  the  various  users.  These  citations  are  then  assigned  to  the 
recipient's  pending  file  (an  electronic  "in-basket").  A pending  file  is 
provided  for  each  office  code  and  for  each  individual.  A user  may  look  at  a 
summary  of  his  pending  file  to  determine  pertinent  information  concerning  the 
messages . 

The  user  may  then  handle  the  message  in  any  manner  he  chooses.  He  may 
print  a copy  for  distribution  to  users  who  are  not  on  the  MME  system,  print  a 
copy  if  he  is  building  a hardcopy  readboard,  or  continue  to  use  the  MME  system 
to: 

(a)  include  the  message  in  an  electronic  readboard; 

(b)  file  the  message  in  one  or  more  office  files  (any  number  of  files  may 
be  created); 

(c)  pass  it  to  another  code  for  information  or  action  with  or  without 
comments  appended; 

(d)  readdress  the  message  to  another  activity  and  release  it  (or  have  it 
released  if  he  doesn't  have  the  proper  authority  - the  command 
determines  which  users  have  release  authority); 

(e)  prepare  a reply  with  automated  aid  from  the  MME  system;  this  aid  may 
include  extracting  from  the  original  or  referenced  messages, 
preparing  address  lists  based  on  the  original  message,  and  formatting 
the  draft  reply: 

(f)  pass  a draft  message  to  other  J3  codes  for  chop  and  comment  and  then 
rewrite  the  message  based  on  appended  comments  that  are  returned; 

(g)  pass  a final  draft  message  to  the  releaser  for  release  and  automatic 
transmittal  to  the  AUTODIN  system;  or 

(h)  prepare  an  internal  memo  in  a manner  similar  to  (e)  above,  chop  it, 
and  "sign"  it  out  in  a manner  similar  to  that  for  a message. 
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All  of  these  functions  may  either  be  performed  personally  by  the  action 
officers  at  the  CRT  terminals , or  with  various  degrees  of  assistance  from 
clerks,  where  typing  or  routine  nutters  are  involved.  Supplemental  paper  for 
handwritten  drafts  or  hard  copy  distribution  can  be  used  as  desired. 

During  the  period  of  Limited  Experimental  Use  (LEU),  the  J3  staff  has  been 
replicating  the  established  paper-processing  procedures  with  the  MME  system. 
With  only  limited  exceptions,  however,  the  MME  system  has  not  yet  been  used  as 
the  primary  method  of  message  handling,  and  only  test  messages  have  been 
released  from  the  MME  system  to  the  LDMX.  Most  of  the  potential  users  have 
taken  advantage  of  this  LEU  period  to  develop  familiarity  with  the  Automated 
Message  Handling  concept  and  functions.  Some,  of  course,  have  adapted  to  it 
well  while  others  have  found  it  difficult  to  use.  The  specific  comments  of 
all  the  users,  however,  are  important  to  the  experiment  in  determining  both 
the  overall  utility  of  the  concept  and  the  value  of  individual  functions. 
Section  8 contains  the  comments  provided  directly  by  CINCPAC  on  the  system's 
use  to  date. 

The  efforts  of  the  developers  to  date  have  been  directed  to  installing  and 
providing  a system  that,  as  a minimum,  will  be  able  to  support  a sufficient 
number  of  users  to  test  the  concept  of  handling  information  within  a major 
segment  of  the  staff  and  will  provide  the  users  with  enough  confidence 
that  the  system  will  be  used  as  the  primary  mode  of  message  processing.  Until 
these  levels  of  use  and  acceptance  are  reached,  there  can  be  little  confidence 
in  any  preliminary  results  of  the  experiment.  The  hardware  and  software 
upgrades  described  previously  are  addressing  the  speed  of  response,  the 
capacity  of  the  system,  and  CINCPAC' s minimum  functional  requirements.  In 
addition,  there  are  installation  and  terminal  problems  that  also  require 
resolution  before  these  minimum  levels  can  be  achieved.  In  particular,  there 
have  been  numerous  "abnormal  terminations"  of  individual  jobs  (almost  1 52  of 
all  jobs  in  October  and  November). 

The  scheduled  relocation  of  the  CINCPAC  Command  Center  has  required  moving 
terminals  and  has  resulted  in  a lack  of  terminals  for  use  by  the  affected 
codes  during  much  of  the  LEU  period.  Problems  encountered  in  the  transmission 
of  the  encrypted  data  to  terminals  outside  the  blockhouse  have  also  caused 
serious  problems  to  certain  high-volume  users. 

It  is  expected  that  the  minimum  requirements  for  the  start  of  Full 
Experimental  Use  (FEU)  by  CINCPAC  (as  specified  in  the  test  plan  [ref  ( f ) ] ) 
will  be  met  after  the  installation  of  SIGMA  Release  2.2  and  the  resolution  of 
the  abnormal  termination  problems  in  January  1979.  The  period  of  FEU  will 
extend  through  the  end  of  the  experiment  in  September  1979,  and  during  this 
time  CINCPAC  J3  will  rely  on  the  MME  system  to  the  fullest  extent  possible. 

The  evaluation  team  will  continue  to  monitor  the  system's  use  and  compare  it 
with  baselines  developed  from  the  earlier  paper  system.  The  team  will  also 
monitor  and  review  the  use  and  contribution  of  the  MME  system  during  command 
center  exercises  and  will  periodically  survey  the  users  to  determine  the 
evolution  of  new  procedures  within  the  staff.  The  team,  in  close  cooperation 
with  CINCPAC  J3,  will  evaluate  the  general  utility  and  specific  functional 
benefits  of  the  MME  system. 
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The  following  summarizes  the  observations  that  have  been  made. 
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(a)  The  hardware  and  software  have  been  undergoing  significant  changes. 

Neither  the  users  nor  the  evaluators  as  of  yet  have  a baseline  system 
to  judge;  neither  has  J3  built  up  sufficient  confidence  in  the  system 
to  rely  on  it  for  full  message-handling  support. 

(b)  A great  deal  of  the  information  (e.g.,  operational  plans,  memos, 

reports)  needed  by  a user  in  preparing  a message  is  not  available  on  « 

the  MME  system.  This  limitation  must  be  considered  when  extrapolat- 
ing the  results  of  the  MME  to  other  staff  environments. 

(c)  The  system  has  been  in  only  a limited  experimental  use  (LEU)  phase. 

During  the  period,  there  have  been  occasional  system  failures  that 
caused  users  to  lose  data.  Further,  there  have  not  been  enough 
terminals  available.  As  a result,  many  users  have  not  converted  all 
of  their  message-processing  tasks  to  the  MME  system. 

(d)  Step-by-step  user  guides  for  various  functions  (releasing  messages, 
reading  incoming  messages,  composing  and  sending  notes)  appear  to  be 
effective  in  encouraging  system  use. 

(e)  The  features  most  used  to  date  have  been  message  routing,  filing,  and 
retrieval.  The  routing  of  messages  to  those  users  who  have  terminals 
has  become  an  almost  automated  routine.  J301  initiates  a series  of 
machine  instructions  to  route  the  traffic,  waits  for  the  MME  system 
to  complete  the  work,  and  then,  still  using  the  MME  system,  routes 
any  remaining  messages.  Using  this  procedure,  the  time  for  routing 
messages  can  be  decreased  from  approximately  four  hours  with  the 
manual  system  to  approximately  one  hour  using  the  MME  system.  The 
major  problem  is  that  all  the  users  served  by  J301's  routing  do  not 
have  terminals;  hence,  the  manual  system  cannot  be  replaced 
completely.  When  filing,  information  copies  of  messages  can  be 
retrieved  by  users  directly  from  the  file  by  means  of  various  message 
selectors.  Thus,  the  information  copies  of  messages  need  not  be 
routed  explicitly  by  J301;  rather,  he  can  rely  on  the  users  using 
their  own  specialized  retrieval  criteria  for  the  messages.  In 
addition,  the  users  utilize  the  file-building  and  message-retrieval 
capabilities  for  their  personal  and  organizational  files. 

(f)  The  command  center  and  joint  reconnaissance  center  (JRC)  watch  teams 
are  using  the  MME  system  to  review  their  traffic  and  to  create  files 
that  reflect  particular  areas  of  concern.  They  then  file  relevant 
messages  in  them  for  use  in  creating  readboards  and  preparing  sections 
of  the  morning  brief. 

(g)  Some  action  officers  are  using  the  MME  system  to  review  incoming 
traffic;  to  retrieve  referenced  messages;  to  create  text  objects  as 
bases  for  sections  of  reports,  memoranda,  and  messages;  and  to 
maintain  large  files  relevant  to  their  duties. 
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(h)  The  clerical  personnel  are  using  the  MME  system  mainly  for  message 
retrieval  because  the  traffic  is  routed  directly  to  the  staff 
officers  for  their  disposal;  they  also  utilize  the  informal  note 
capability. 

(i)  The  strict  enforcement  of  a security  policy  using  the  concept  of  a 
security  kernel  does  not  appear  to  have  added  undue  restrictions  on 
the  user  interface. 


It  is  interesting,  although  not  surprising,  that  the  functions  generally 
considered  most  desirable  at  this  i.ae  (filing  and  initial  message 
distribution)  are  the  only  ones  which  have  been  exercised  to  any  great  extent 
during  the  LEU  period.  The  transition  to  primary  use  of  the  automated  system 
should  lead  to  a better  understanding  of  the  advantages  and  disadvantages  of 
the  concept  and  allow  a more  thorough  evaluation  of  its  utility. 

In  conclusion,  all  of  the  objectives  outlined  in  the  MOA  are  being 
addressed  by  the  experiment.  However,  with  the  exception  of  multi-level 
security  where  most  studies  have  been  completed,  the  use  of  the  system  as 
CINCPAC  J3's  primary  message  building  tool  is  necessary  to  substantiate  the 
observations  and  fully  determine  the  impact  of  the  MME  concept. 

The  remainder  of  this  report  provides  more  detail  on  the  experiment  set 
up,  training,  statistical  survey  results,  and  user  comments.  More  information 
is  also  provided  in  the  referenced  documents  and  reports. 


SECTION  2 


BACKGROUND 


The  military  message  experiment  (MME)  was  conceived  to  determine  the  need 
for  and  type  of  further  automation  in  the  handling  of  military  messages.  The 
parallels  between  the  user  services  provided  by  the  messaje  processing  systems 
being  developed  by  the  Deferse  Advanced  Research  Projects  Agency  (D\RPA) 
contractors  and  the  user  requirements  within  military  staffs  led  to  a 
Memorandum  of  Agreement  [reference  (a)]  between  the  Director,  Defens®  Advanceu 
Research  Projects  Agency;  the  Commander,  Naval  Telecommunications  Command;  the 
Commander,  Naval  Electronics  Systems  Command;  and  the  Commander  in  Chief, 
Pacific.  The  original  MOA  was  signed  in  December  1975,  and  the  current 
version  was  signed  in  September  1978.  Under  the  MOA,  DARPA  has  general 
responsibility  for  the  development  of  the  MME  system;  COMNAVTELCOM  acts  as  the 
single  point  of  contact  for  the  Navy;  COMNAVELEXSYSCOM  has  general  respons- 
ibility for  evaluation  of  the  MME;  and  CINCPAC  has  general  responsibility  for 
providing  facilities,  services,  personnel,  and  support  functions. 

The  basic  elements  of  the  MME  include  the  following: 

(a)  a PDP-10  computer  with  the  TENEX  operating  system  has  been  installed 
in  a TOP  SECRET  facility  at  CINCPAC  and  dedicated  to  running  me.'tsage- 
handling  software  for  the  MME  evaluation;  it  has  an  on-line 
connection  to  the  AUTODIN  system  via  the  Local  Digital  Message 
Exchange  (LDMX); 

(b)  DARPA  contractors  have  developed  and  installed  a message  service 
system  on  Che  PDP-10,  a terminal  interface  processor  (PDP-11),  and  a 
set  of  user  terminals; 

(c)  members  of  the  Operations  Directorate  (J3)  at  CINCPAC  Headquarters, 
Camp  Smith,  Hawaii,  are  using  the  terminals  and  computer  system  for 
receiving  and  filing  messages.  With  the  installation  and  checkout  of 
revised  coordination  and  release  procedures,  the  system  will  be  used 
for  generation,  release,  and  transmission  of  messages;  this  use  will 
continue  through  September  1979;  and 

(d)  the  results  of  the  MME  evaluation  will  be  used  to  influence  the 
specification  and  design  of  production  hardware  and  software  for 
future  message  handling  systems  in  DoD. 

GOALS 

The  primary  objective  of  the  MME  is  to  determine  the  utility  of  an 
interactive  message  service  in  a major  military  headquarters.  As  a part  of 
this  determination,  alternative  features  and  capabilities  must  be  identified. 
These  requirements  will  be  used  as  a baseline  for  developing  Automated  Message 
Handling  Systems  for  military  use.  Accordingly,  the  following  specific 
objectives  have  been  identified  in  the  MOA: 
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(a)  determine  and  demonstrate  the  usefulness  of  automated  message 
capabilities  and  the  necessary  features  to  support  a military  message 
handling  system  in  an  operational  environment; 

(b)  determine  the  effect  of  an  automated  message  handling  system  on 
operational  procedures,  manpower,  and  logistics  in  an  operational 
environment ; 

(c)  determine  the  training  requirements  associated  with  automated  message 
handling  systems; 

(d)  determine  the  characteristics  of  an  acceptable  user  interface  for  an 
interactive  automated  message  handling  system; 

(e)  determine  multi-level  security  design  characteristics  and  their 
impact  on  the  user  interface;  and 

(f)  obtain  the  data  necessary  to  assist  in  the  future  design  and 
development  of  a family  of  automated  message  handling  systems  for  DOD 
use. 

During  the  period  covered  by  this  report,  the  MME  system  has  been  used 
only  on  a limited  basis.  Full  experimental  use  of  the  system  will  begin  after 
the  minimum  requirements  listed  in  the  test  plan  [reference  (f)]  are  met. 

Even  though  the  experiment  has  not  yet  started,  it  may  be  useful  to  review  the 
objectives  against  the  information  gained  during  the  LEU.  Any  strong  inference 
concerning  objective  (a)  would  be  premature.  The  system  is  certainly  being 
used,  but  it  is  too  early  to  claim  that  the  usefulness  has  been  demonstrated. 
The  same  is  true  of  (b)  and  ( c ) ; although  a substantial  training  effort  has 
been  expended,  the  results  aren't  certain  until  the  trained  users  begin  to  use 
the  system  as  a part  of  their  day-to-day  routine.  The  security  design  appears 
to  have  provided  an  effective  user  interface  (objective  (d)  and  (e)),  but  this 
observation,  like  all  others,  is  subject  to  confirmation  during  the  experiment. 
During  the  selection  of  the  message  system  for  the  MME,  the  design  of  the 
security  interface,  and  specifications  for  the  changes  to  SIGMA,  much  of  the 
information  needed  to  assist  in  the  development  of  future  message  systems 
(objective  (f))  has  been  obtained.  The  value  of  this  information,  however,  is 
dependent  on  the  determination  of  the  usefulness  of  the  MME  concept  at  CINCPAC. 

EXPERIMENT  HISTORY 

The  impetus  for  the  MME  was  an  observation  by  Congress  that  there  were 
numerous,  apparently  uncoordinated,  message  center  developments  by  the 
Military  Departments.  This  resulted  in  a memo  from  the  Director,  Telecom- 
munications and  Command  and  Control,  OSD,  on  26  June  1975,  directing  that 
techniques  needed  for  secure  interactive  message  systems  be  developed.  As  a 
result,  the  principals  involved  with  the  MME  signed  a Memorandum  of  Agreement 
in  December  1975,  leading  to  the  development  of  an  experiment  to  validate  the 
message  processing  requirements  for  all  services. 

At  the  time  the  MOA  was  signed,  there  were  two  efforts  funded  to  develop  a 
military  version  of  an  ARPANET  message  system.  Bolt,  Beranek,  and  Newman  of 
Cambridge,  Massachusetts,  was  working  on  a system  that  led  to  HERMES,  and  the 
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Information  Sciences  Institute  of  the  University  of  Southern  California  (ISI) 
was  working  on  what  was  to  become  SIGMA.  An  additional  message  system  had 
been  developed  by  MIT  under  other  ARPA  funding,  and  it,  also,  was  considered 
as  a candidate  for  the  experimental  system. 

Preliminary  designs  for  the  "militarized"  versions  of  the  message  systems 
were  generated  and  discussed  during  the  Spring  of  1976.  During  a design 
review  meeting  at  CINCPAC  in  the  Summer  of  1976,  it  was  concluded  that  the 
implementations  of  the  security  policy  were  resulting  in  an  unacceptable  user 
interface.  The  rigid  enforcement  of  some  rules  was  eased,  and  the  three 
organizations  redesigned  the  security  enforcement  mechanisms.  From  22 
February  1977  to  3 March  1977,  a formal  evaluation  of  the  three  systems  was 
conducted  at  BBN,  Cambridge. 

The  evaluators  concluded  that  the  performance  of  HERMES  was  good,  but  that 
it  lacked  the  features  needed  in  the  military  environment,  that  SIGMA 
presented  the  user  with  an  interface  and  features  that  would  allow  the  most 
useful  data  to  be  derived  from  the  system,  and  that  the  performance  of  the  MIT 
system,  DMS,  was  too  marginal  at  the  time  of  the  evaluation  to  risk 
installation  in  a military  environment.  The  evaluators  recommended  that  SIGMA 
be  chosen  as  the  experimental  system,  but  noted  there  was  a considerable  risk 
in  upgrading  the  performance  of  SIGMA  to  an  acceptable  degree.  The  selection 
procedures  are  documented  in  references  (c)-(e)  and  (g)-(i). 

SIGMA  was  chosen  as  the  message  system,  and  a plan  was  developed  to 
correct  the  known  SIGMA  deficiencies.  An  additional  256K  of  core  memory  was 
ordered  for  the  PDP-10.  SIGMA  was  installed  for  use  at  CINCPAC  in  May  1977. 
The  performance  was  unacceptable,  and  a group  was  formed  to  examine  SIGMA  and 
recommend  changes  to  improve  the  performance.  During  this  study,  an  updated 
SIGMA  was  installed  in  December  1977,  and  it  also  resulted  in  a system  that 
was  not  operationally  useful. 

The  entire  experiment  was  reevaluated,  a configuration  control  system  for 
SIGMA  releases  was  devised,  another  hardware  upgrade  (KL  processor  and 
additional  312K  words  of  memory)  was  defined,  and  a new  schedule  with 
acceptance  critieria  was  developed. 

SIGMA  Release  2.0  was  installed  in  June  1978.  A period  of  limited 
experimental  use  which  supported  10-15  users  on-line  began  in  July  1978;  the 
response  was  generally  acceptable  under  light  user  load.  Changes  to  increase 
responsiveness  and  functionality  were  made  to  the  software,  and  SIGMA  2.1  was 
installed  in  September  1978.  The  hardware  was  upgraded  from  a KA  to  a 
more-powerful  KL  processor  in  October  1978.  The  memory  was  increased  from  the 
original  262,144  words  to  786,432  to  the  final  million-word  (1,048,576)  system 
in  October  1978. 

The  following  software  upgrades  are  planned  for  the  remainder  of  the 
experiment. 

SIGMA  2.2,  January  1979,  will  provide  the  improved  capabilities  needed  for 

the  coordinate/release  function,  increased  responsiveness,  and  the 

addition  of  a few  user  functions. 
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SIGMA  2.3,  April  1979,  will  provide  discretionary  access  controls  and  some 
additional  user  functions. 

SIGMA  2.4,  July  1979,  is  the  last  planned  upgrade.  It  will  provide 
improved  file  archiving  capabilities  and  some  additional  user  functions. 

The  only  additional  hardware  upgrade  planned  is  to  complete  the 
installation  of  all  the  terminals.  The  addition  of  better-quality  printers  is 
being  considered. 


SECTION  3 


MANUAL  MESSAGE  HANDLING 


The  manual  message  processing  methods  in  use  within  the  Operations 
Directorate  (J3)  prior  to  the  introduction  of  the  MME  System  are  documented  in 
the  Baseline  Data  Report  [reference  (j)].  This  section  is  a summary  provided 
by  the  author  of  reference  (j).  The  information  was  obtained  by 
questionnaires,  checksheets,  observation,  interviews,  and  examination  of 
records.  The  bulk  of  the  data  was  collected  between  March  and  April  1977,  and 
in  April  1978. 

OVERVIEW  OF  MANUAL  PROCEDURES 
Incoming  Messages 

Messages  are  sent  to  CINCPAC  via  the  AUTODIN  system.  They  are  received  by 
the  Local  Digital  Message  Exchange  (LDMX)  in  the  Telecommunications  Center. 
Messages  of  immediate  or  higher  precedence  are  routed  immediately  by  the  LDMX 
to  a printer  in  the  Command  Center.  The  LDMX  identifies  an  action  Directorate 
for  each  message.  At  the  Communications  Center,  copies  of  the  messages  are 
made  and  sorted  for  pickup  by  clerks  from  the  Directorates  designated  to 
receive  the  messages.  During  duty  hours,  the  Communications  Center  notifies 
the  Adminstrative  Section  of  the  action  Directorate  when  a message  of 
immediate  or  higher  precedence  is  received. 

Within  the  Operations  Directorate,  the  Administrative  Section  (J301)  sorts 
the  messages  and  assigns  responsibility  for  each  action  message.  If  a message 
deals  with  a topic  in  which  an  office  is  known  to  have  special  interest  or 
expertise,  the  message  is  assigned  directly  to  that  office.  Copies  of  action 
and  some  information  messages  are  put  on  readboards  for  the  Director  and  his 
Deputy.  Messages  are  held  for  pickup  by  some  divisions  and  delivered  to 
others. 

In  some  divisions,  messages  are  reviewed  by  the  division  chief,  who 
assigns  an  action  branch  or  office  to  each  of  that  division's  action  messages. 
In  other  divisions,  action  responsibility  is  almost  always  assigned  by  J301; 
then  these  messages  are  delivered  to  the  people  responsible  for  them.  In  some 
divisions,  message  copies  are  also  kept  on  readboards  in  the  division  office 
for  review  by  the  officers. 

An  officer  receives  a copy  of  any  message  for  which  he  has  been  assigned 
action  responsibility.  His  copy  of  the  message  may  include  annotations  by  his 
Director  or  his  division  chief.  The  action  officer  may  respond  by  creating 
another  message,  by  providing  information  to  answer  a question,  or  by  filing 
the  message  in  a project  file.  The  officers  may  also  need  to  retrieve  old 
messages  for  use  as  references  to  an  outgoing  message  or  to  serve  as 
background  for  their  decision  making. 

At  the  division  and  branch  level,  the  clerks  and  administrative  clerks 
distribute,  file,  and  retrieve  incoming  messages. 
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Outgoing  Messages 

An  officer  with  responsibility  for  preparing  an  outgoing  message  from 
CINCPAC  carries  out  whatever  research  is  necessary  by  consulting  files  of 
related  messages  and  documents.  He  drafts  the  message  and  usually  has  it 
typed  by  a division  clerk.  After  proofreading  and  correcting  the  message,  he 
coordinates  the  draft  with  his  colleagues  and  his  immediate  superior.  The 
initial  coordination  will  often  be  done  face-to-face.  Several  retypings  may 
be  necessary  if  changes  are  recommended  by  each  coordinator.  Coordination  of 
the  final  OCR-readable  copy  of  the  message  is  carried  out  by  each  coordinator 
initialing  the  first  page  of  the  message. 

When  the  message  is  ready  for  release,  the  final  copy,  with  a staff 
sunmary  and/or  copies  of  relevant  backup  material,  is  given  to  the  releasing 
officer  for  his  signature.  If  he  approves  transmission  of  the  message,  the 
final  copy  is  carried  to  the  Communications  Center.  A comeback  copy  of  the 
transmitted  message  will  later  be  returned  to  the  originating  office  and  to 
other  directorates  designated  in  the  drafter  distribution  field  on  the  form 
sent  to  the  Communications  Center. 

Command  Center  Message  Handling 

At  the  same  time  that  messages  are  going  through  the  normal  routing 
process,  the  Duty  Director  of  Operations  (DDO)  in  the  Command  Center  monitors 
all  messages  received  by  CINCPAC.  If  he  receives  an  important  message  during 
duty  hours,  he  may  notify  the  appropriate  office  immediately  as  a back-up  or 
follow-up  to  the  Communications  Center  procedure.  After  duty  hours,  the  DDO 
notifies  the  appropriate  Directorate  duty  officer  of  a high  precedence 
message.  The  DDO  also  selects  some  messages  for  the  Director's  readboard. 

t A build-up  of  activity  in  a sensitive  area  of  the  PACOM,  whether  signaled 

by  high  precedence  messages  or  telephone  calls,  may  cause  the  DDO  to  recommend 
that  the  J3  initiate  crisis  (or  exercise)  operations.  At  these  times,  the 
Conmand  Center  becomes  the  center  of  operations.  Special  teams  are  brought  in 
to  handle  the  crisis,  while  the  Command  Center  Watch  Team  (CCWT)  continues  to 
monitor  non-crisis  or  non-exercise  activities  and  to  provide  support  to  the 
crisis/exercise  management  teams. 

DISTRIBUTION 

From  the  time  it  is  received  by  the  LDMX  to  the  time  it  is  delivered  to 
the  division  or  branch  for  use  by  the  action  officer,  an  incoming  message 
passes  through  several  phases  of  distribution.  Depending  on  its  precedence 
and  time  of  arrival,  a message  may  take  from  one  minute  to  six  hours  to  be 
processed  in  the  Communications  Center.  On  the  average,  routine  messages  are 
processed  in  76  minutes,  priority  messages  are  processed  in  58  minutes, 
immediate  messages  are  processed  in  24  minutes,  and  higher  precedence  messages 
are  processed  in  five  minutes  or  less.  The  next  step  is  for  the  message  to  be 
picked  up  in  the  Communications  Center  and  processed  at  the  Directorate 
Administrative  Section.  Priority  messages  wait  an  average  of  30  minutes, 

, while  routine  messages  wait  an  average  of  80  minutes  to  be  picked  up  from  the 


Communications  Center.  (Immediate  and  higher  precedence  messages  are  handled 
by  a telephone  call  by  the  CCWT,  so  further  processing  by  the  Communications 
Center  is  not  relevant  to  the  time  it  takes  an  officer  to  see  the  message  the 
first  time.) 

Most  of  the  message  distribution  effort  is  spent  in  the  Communications 
Center  where  message  copies  are  prepared  for  distribution,  and  in  directorate 
administrative  sections  where  the  initial  decisions  are  made  on  action 
assignment  and  information  message  routing.  Some  effort  is  spent  on  , 

distribution  in  the  division  and  branch  offices,  but  this  is  proportionally 
less  than  that  spent  in  the  directorate  office. 

Observation  of  the  staff  in  J301  showed  that  they  process  a batch  of 
messages,  on  the  average,  in  25  minutes.  This  includes  time  for  sorting  the 
batch,  assigning  action,  preparing  the  messages  for  pickup  or  delivery,  and 
preliminary  filing.  Delivery  to  the  division  or  branch  takes  an  additional  10 
minutes. 

The  age  of  the  message  when  it  reaches  the  action  officer  is  related  to 
precedence  as  well  as  to  the  time  of  day  it  is  printed  by  the  LDMX.  Priority 
messages  average  two  hours  from  delivery  at  the  LDMX  to  delivery  at  the 
division  or  branch,  and  routine  messages  take  an  average  of  three  hours  to  be 
processed  and  delivered.  Some  divisions  do  not  have  messages  delivered  by 
J301,  but  send  their  own  runners  to  the  admin  section  to  pick  up  messages.  In 
these  divisions,  the  messages  would  be  still  older  upon  receipt.  See  Table  1. 

The  message  load  for  the  J3  Directorate  is  fairly  stable,  averaging  about 
2000  mess£?es  a week.  At  the  division  and  branch  level  the  load  is  relatively 
stable  for  a particular  division  or  branch,  but  varies  considerably  from 
division  to  division  and  from  branch  to  branch.  Thus,  the  amount  of  effort  a 
clerk  spends  or.  message  distribution  depends  on  the  division  and  branch  that 
he  supports. 

In  the  Command  Center,  the  clerk  spends  some  of  his  time  monitoring  the 
printer  and  distributing  these  messages  to  the  duty  officers.  He  spends,  a 
higher  proportion  of  his  time  on  message  distribution  than  do  the  division  and 
branch  clerks. 


MESSAGE  REVIEW 

A message  provides  information  for  action  officers,  who  may  need  to 
respond  to  the  message  or  to  save  it  for  future  reference.  The  amount  of  time 
spent  on  message  review  varies  according  to  the  message  load.  Officers  in 
branches  with  heavy  loads  review  an  average  of  11  action  messages  and  39  info 
messages  daily;  they  spend  about  45  minutes  on  this  review.  Officers  in  some 
of  the  other  branches  receive  very  few  action  messages.  They  spend  about  25 
minutes  daily  reviewing  an  average  of  50  info  messages.  In  branches  with 
light  loads,  only  a few  action  and  info  messages  are  received,  and  about  five 
minutes  are  spent  reviewing  incoming  messages. 


Table  1.  Manual  Message  Processing 


Routine 

Priority 

Immedidate 

Higher 

Precedence 

Communications 

Center  Processing 

76  min. 

58  min. 

24  min. 

5 min. 
or  less 

Waiting  Time 
for  Pickup 

80  min. 

30  min. 

Note  1 

Note  1 

Total  Average 
delay  from 
printout  by  LDMX 
to  receipt  by 

Action  Officer 

3 hrs. 

2 hrs 

Note  2 


Note  1 - Notification  of  receipt  of  messages  of  immediate  or  higher  precedence 
messages  is  made  to  the  appropriate  action  officer  by  the  CCWT  by 
telephone. 

Note  2 - This  depends  also  on  time  of  day  of  message  rereipt  in  the 

Communications  Center.  The  delay  time  includes  processing  in  J301 
and  delivery  to  division/branch. 


In  the  Command  Center,  message  review  is  affected  by  the  personal  style  of 
the  duty  officers,  as  well  as  by  the  assignment,  or  desk.  During  the  period 
that  baseline  data  were  being  collected,  the  DDOs  averaged  40  minutes  of 
message  review  per  shift,  the  Surface  Ops  duty  officers  averaged  around  80 
minutes,  and  the  Air  Ops  duty  officers  averaged  120  minutes.  The  Air  Ops  duty 
officer  has  the  responsibility  for  creating  the  Director's  readboard;  thus,  he 
spends  the  most  time  on  message  review. 
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MESSAGE  FILING  AND  RETRIEVAL 


Messages  are  kept  on  disk  memory  in  the  LDMX  for  12  to  15  days,  are 
retained  on  magnetic  tape  in  the  Communications  Center  for  90  days,  and  are 
stored  in  the  J304  vault  in  microfilm  files  for  several  years.  In  the  admin 
section  (J301),  xerox  copies  of  the  messages  are  kept  in  file  boxes  for  30 
days.  When  a message  is  needed  but  cannot  be  found  in  division  or  branch 
files,  a copy  is  requested  from  J301.  If  the  message  is  more  than  30  days 
old,  it  is  requested  from  the  Communications  Center,  if  more  than  90  days  old, 
from  J304. 

In  J301,  preliminary  filing  is  done  in  conjunction  with  the  message 
sorting  for  distribution.  Final  filing  (by  minute  of  the  message's 
date-time-group)  is  done  on  the  night  shift.  During  the  regular  workday, 
filing  takes  only  a few  minutes  of  the  day.  J301  clerks  receive  only  a few 
requests  each  day  for  messages  in  their  files.  These  can  usually  be  satisfied 
in  five  minutes  or  less. 

In  the  divisions  and  branches,  the  clerical  personnel  handle  most  of  the 
filing  for  the  officers.  Although  filing  may  not  be  done  every  day,  it 
generally  takes  about  half  an  hour  of  the  clerk's  time  when  it  is  done. 

Filing  may  accumulate  over  several  days,  or  occur  as  a project  is  finished. 
Relatively  less  retrieval  is  done,  since  messages  of  current  interest  are  kept 
at  the  officer's  work  place,  and  do  not  need  to  be  retrieved  from  the  branch 
files.  In  addition,  the  action  officers  do  relatively  more  retrieval  than 
filing,  which  suggests  that  when  they  need  something  they  find  it  for 
themselves.  For  example,  on  days  when  retrieval  occurred,  officers  in 
branches  with  heavy  incoming  message  loads  spent  an  average  of  30  minutes 
retrieving  messages;  in  branches  with  moderate  loads  they  spend  15  minutes; 
and  in  branches  with  light  loads,  they  spent  10  minutes. 

The  pattern  in  the  Command  Center  is  similar,  with  the  clerks  doing  most 
of  the  filing  and  relatively  little  retrieval.  Although  the  duty  officers  do 
some  retrieval,  they  also  keep  messages  of  current  interest  at  hand,  and 
hence,  do  not  usually  need  to  retrieve  messages  from  long-term  storage.  When 
they  do  need  such  retrieval,  they  often  recall  the  message  from  the 
Communications  Center,  a task  thay  can  accomplish  with  a telephone  call.  The 
message  is  delivered  directly  to  the  Command  Center  via  a pneumatic  tube. 

MESSAGE  CREATION 

Messages  are  formal  record  communications.  They  are  drafted  in  response 
to  other  messages,  in  response  to  non-message  communications,  or  in  order  to 
initiate  an  action  or  project.  Letters,  memos,  etc.  may  be  drafted  instead  of 
formal  messages. 

The  number  of  messages,  letters,  etc.,  which  are  drafted  by  action 
officers  does  not  vary  greatly  among  branches  with  varying  incoming  message 
loads.  Although  the  data  do  suggest  that  officers  in  branches  with  light 
incoming  message  loads  create  more  outgoing  communications  than  the  other 
officers,  even  these  officers  create  only  about  two  items  per  day,  and  often 
have  days  when  no  message,  letter,  or  memo  creation  occurs. 
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Creation  is  a relatively  time-consuming  task.  The  originator  must  gather 
background  information  and  organize  the  material  in  addition  to  composing  the 
message.  Although  some  messages,  letters,  and  memos  are  routine  requiring 
little  effort,  and  some  are  difficult  lengthy  items  taking  several  hours  to 
write,  most  take  about  an  hour  of  the  officer's  time. 

Typing  these  communications  is  also  a relatively  time-consuming  task  for 
the  clerical  personnel.  Because  each  clerk  supports  several  action  officers, 
the  clerk  handles  a higher  number  of  outgoing  items.  Overall,  clerks  type 
(and  retype)  an  average  of  four  outgoing  items  daily.  The  amount  of  time 
taken  depends  on  the  length  of  the  items;  but  on  the  average,  the  clerks  spend 
75  minutes  daily  on  typing  tasks. 

Fever  outgoing  messages,  letters,  and  memos  are  sent  from  the  Command 
Center;  some  of  the  messages  are  transmitted  over  the  WWMCCS  system  to  the 
LDMX  and  AUTODIN.  On  the  average,  Command  Center  duty  officers  send  a total 
of  about  four  outgoing  items  per  shift,  and  they  spend  an  average  of  40 
minutes  on  creation  tasks.  The  clerks,  perhaps  because  they  are  using  the 
WWMCCS  terminals  which  provide  on-line  editing  capabilities,  spend  an  average 
of  only  18  minutes  a day  typing  and  retyping  outgoing  communications. 


COORDINATION  AND  RELEASE 

Before  a message  can  be  released  and  transmitted,  it  must  be  coordinated 
to  ensure  accuracy  and  completeness.  After  all  interested  parties  have 
approved  the  message,  it  is  sent  to  the  Director  or  a designated 
representative  for  formal  release.  Only  after  the  message  has  been  released 
can  it  be  transmitted.  Most  messages  are  seen  by  three  to  five  coordinators 
before  they  are  sent  for  release. 

The  number  of  messages  that  action  officers  receive  for  coordination 
varies  considerably  from  position  to  position.  Some  officers  may  receive  only 
one  message  a week,  others  receive  one  or  two  a day.  Branch  chiefs  typically 
receive  the  most  messages;  they  may  review  these  themselves  or  they  may  give 
them  to  a member  of  the  branch  to  review. 

On  days  when  messages  are  received  for  coordination,  the  time  spent 
reviewing  and  commenting  on  the  message  may  range  from  five  minutes  for  a 
routine  message  to  an  hour  or  more  for  a long  complicated  message.  On  the 
average,  around  25  minutes  are  spent  coordinating  messages.  Officers  who 
receive  the  greatest  number  of  incoming  messages  are  the  least  involved  in 
coordinating  outgoing  messages,  letters,  and  memos. 

Fewer  messages  are  received  by  Command  Center  duty  officers  for  their 
coordination.  When  they  do  receive  messages,  they  average  around  ten  minutes 
on  coordination  tasks. 

Messages  are  released  by  the  J3,  the  Director  of  Operations,  or  by  his 
representatives  the  J3  Deputy,  a division  head,  or  the  DDO.  Message  release 
is  usually  a routine  task  which  takes  only  a few  minutes  for  each  message. 

The  releaser  relies  on  the  preparation  and  review  already  done  by  the 
originator  and  the  coordinators. 


TRANSMISSION  AND  POST- TRANSMISSION 


Once  released,  the  outgoing  message  is  sent  to  the  Communications  Center 
for  transmission  over  AUTODIN.  (Memos  are  circulated  within  CINCPAC;  letters 
are  mailed.)  At  the  Communications  Center  the  message  is  checked  for  format, 
addressee  list,  releaser's  signature,  etc.  Flash  and  immediate  messages  are 
processed  directly.  Priority  and  routine  messages  are  put  in  a tray  for 
processing  as  soon  as  a clerk  has  time. 

The  average  processing  time  for  priority  messages  is  76  minutes  from  time 
of  receipt  in  the  Communications  Center  to  time  of  transmission.  For  routine 
messages,  the  average  processing  time  is  131  minutes.  These  times  vary,  of 
course,  according  to  the  time  of  day  and  the  outgoing  message  load.  Comeback 
copies  are  processed  and  delivered  along  with  the  incoming  messages. 


COMMENT 

In  J3,  a person's  message-handling  activities  vary  according  to  his 
position  and  to  his  division  or  branch.  Activities  also  vary  from  day  to  day, 
depending  on  current  assignments.  It  would  be  possible  for  a clerk  to  spend 
all  of  a day  distributing,  filing,  and  typing  messages.  It  would  be  possible 
for  an  action  officer  to  spend  all  of  a day  reading,  retrieving,  creating,  and 
reviewing  messages.  This  is  not  usually  the  case;  on  most  days  personnel 
spend  less  than  half  their  day  on  message-handling  activities.  In  fact,  on 
many  days,  many  officers  are  not  involved  in  handling  outgoing  messages  at 
all.  Incoming  messages  far  outnumber  outgoing  messages;  however,  in  a 
message-by-message  comparison,  outgoing  messages  take  relatively  more  time  to 
handle. 

It  is  important  to  note  that  messages  are  only  one  form  of  written 
communication  used  in  J3.  Letters,  memos,  and  other  documents  are  also  used. 
Thus,  to  help  personnel  with  their  complete  job,  the  MME  should  examine  the 
role  of  automation  in  the  total  communication  system  of  the  staff. 
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SECTION  4 


AUTOMATED  MESSAGE  HANDLING 


There  are  clear  parallels  in  the  automated  and  the  manual  system.  Some  of 
the  parallels  are  deliberate  - the  automation  of  existing  successful  manual 
procedures;  some  of  the  parallels  are  necessary  because  the  manual  system  will 
remain  as  a backup  and  because  all  the  divisions  in  the  Operations  Directorate 
will  not  be  automated  during  the  experiment. 

INCOMING  MESSAGES 

As  with  the  manual  system,  formal  messages  for  CINCPAC  are  received  at  the 
Communications  Center  LDMX.  The  LDMX  routing  algorithm  sends  a copy  of  the 
messages  destined  for  the  Operations  Directorate  or  the  Command  Center  to  the 
MME  system  via  the  electrical  interface.  Thus  the  waiting  time  for  the 
messages  distributed  to  J3  to  arrive  in  the  Directorate  Administrative  Section 
(J301)  is  reduced  from  the  average  156  minutes  for  routine  (88  minutes  for 
priority  and  24  minutes  for  immediate)  to  a fraction  of  that  time.  The 
messages  are  delivered  to  J301's  electronic  in-basket  for  proper  assignment  to 
J3  action  officers. 

The  MME  system  provides  the  person  performing  the  J301  distribution 
function  with  a number  of  aids.  These  are  discussed  more  fully  in  the  next 
section  describing  the  MME  system  software  functions.  By  using  these  aids, 
J301  sorts  out,  collects,  and  distributes  messages  electronically  to  those 
divisions  participating  in  the  experiment.  The  handling  of  multiple  paper 
copies  of  messages  is  reduced,  and  the  J301-routing  function  is  performed  in  a 
fraction  of  the  time  required  for  the  manual  system. 

The  action  assignment  by  J301  results  in  the  message  being  delivered  to 
the  electronic  in-baskets  for  the  action  officers  and  watch  officers.  In 
addition  to  being  routed  to  J301  for  further  distribution,  messages  are  also 
filed  in  what  is  called  a datefile  that  currently  is  accessible  by  any 
authorized  MME  system  user  with  the  proper  clearance.  With  the  implementation 
of  SIGMA  release  2.3,  discretionary  access  controls  will  be  provided  so  that 
any  J3-decreed  need-to-know  restrictions  can  be  enforced  on  the  datefile. 

In  addition  to  acting  on  the  action  messages  assigned  to  them  by  J301, 
many  action  and  watch  officers  have  created  sophisticated  selectors  (retrieval 
aids)  with  which  they  search  the  datefile  for  messages  of  interest.  Once  the 
action  and  watch  officers  receive  messages  (either  by  assignment  of  messages 
from  J301  or  by  screening  the  datefile),  they  take  appropriate  action.  This 
action  includes  filing  the  message  in  a user  file,  forwarding  the  message, 
generating  a reply  (at  this  point  in  the  experiment,  outgoing  messages  are 
handled  manually),  or  deleting  the  message  from  the  action  officer's  file  (he 
cannot  delete  it  from  the  master  file). 
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MESSAGE  REVIEW 


The  MME  system  presents  messages  for  action  officer  review  on  the 
terminal.  In  addition  to  the  capability  of  reading  message  text,  the  MME 
system  allows  the  user  to  look  at  files  of  message  citations.  The  proper 
content  of  a message  citation  is  still  the  subject  of  experimentation  (see 
respondent  #8's  comments  concerning  scanning  (section  8)).  Currently  the 
citation  contains  the  ORIG,  DTG,  SUBJ,  and  first  line  of  text. 

Because  many  users  believe  that  scanning  messages  on  paper  is  faster, 
consideration  is  being  given  to  providing  printers  so  that  hard  copy  can  be 
produced  for  the  sole  purpose  of  fast  scanning. 


MESSAGE  FILING  AND  RETRIEVAL 

Formal  (AUTODIN- type)  messages  in  the  MME  are  never  deleted  or  destroyed 
intentionally.  Little-used  messages  are  archived  to  tape,  but  the  only  effect 
on  users  is  an  increased  retrieval  time  for  these  messages.  The  files  in 
which  users  ''file  messages"  are  not  really  message  files,  but  are  files  of 
citations.  Only  one  copy  of  each  message  is  retained  in  the  system,  and  it  is 
in  the  master  file.  Thus,  many  users  can  "have"  a long  message  in  their 
files,  but  there  in  fact  exists  only  one  copy  of  the  message. 

The  users  currently  make  use  of  the  filing  and  retrieval  capability,  but 
the  system  has  not  been  in  use  long  enough  to  evaluate  the  efficacy  of  the  use 
of  the  files. 


MESSAGE  CREATION,  COORDINATION,  AND  RELEASE 

The  MME  system  has  been  used  only  for  generating  test  messages.  The 
proper  coordination  facilities  were  not  available  in  the  system  described  in 
this  report.  It  appears  that  such  coordination  capability  is  needed  to  allow 
action  officers  the  flexibility  they  desire  for  coordinating  and  releasing 
messages.  This  capability  is  scheduled  for  release  2.2  (Jan  1979). 


TRANSMISSION  AND  POST- TRANSMISSION 

After  a message  is  released  on  the  MME  system,  it  is  transmitted  over  the 
electrical  interface  to  the  LDMX.  The  LDMX  provides  a "come-back"  copy  over 
the  link  to  the  MME  system  after  the  message  is  transmitted. 


AUTOMATED  MESSAGE  HANDLING  SYSTEM 

The  system  used  to  provide  these  automatic  capabilities  consists  of  three 
basic  hardware  subsystems:  the  central  processor  subsystem,  the  interface 
subsystem,  and  the  terminal  subsystem.  The  central  processor  subsystem 
onsists  of  a PDP-10  with  one  million  words  of  memory  and  a KL-10  processor. 
The  interface  subsystem  provides  the  connection  between  the  MME  and  the  LDMX, 
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and  it  links  Che  user  terminals  wich  the  central  processor  subsystem.  The  MME 
terminal  is  a Hewlett-Packard  2649A  with  minor  physical  modifications,  TEMPEST 
modifications,  and  special  firmware  developed  by  ISI.  The  firmware  is 
contained  in  programmable  read-only  memories  (PROMs)  which  mount  on  special 
boards  that  replace  the  HP  control  memory.  As  of  30  November  1978,  16  were 
available  for  use. 

System  Software 


Software  in  the  PDP-10  is  divided  into  three  major  components:  the  TENEX 
operating  system,  the  MME/LDMX  interface  software,  and  the  SIGMA  message 
system  software.  The  users  of  the  system  interface  directly  with  SIGMA  and 
their  view  of  the  system's  utility  is  dependent  primarily  on  SIGM^.  For  the 
most  part,  they  are  not  concerned  with  details  of  the  operating  system  or  the 
MME/LDMX  interface  software.  Thus,  the  following  description  of  the  software 
includes  only  SIGMA  and  describes  those  features  that  most  affect  the  user. 

The  message  coordination  method  is  being  revised;  a new  coordination  method  is 
scheduled  for  SIGMA  Release  2.2  in  January  1979. 

User  Interface 


The  approach  chosen  to  provide  the  necessary  support  for  the  user  who  is 
neither  a trained  operator  nor  a computer  specialist  is  to  interface  him  to 
the  message  service  through  an  "intelligent  front-end  process"  which  is  called 
his  "Agent."  This  Agent  makes  the  service  appear  consistent  to  the  user.  It 
is  designed  to  handle  all  control  procedures  (e.g.,  editing,  help,  defaulting, 
error  handling,  context  mechanisms)  with  the  same  code  and,  therefore,  in  the 
same  way  throughout  all  phases  of  the  service.  Lack  of  this  consistency  has 
been  a major  source  of  difficulty  in  many  computer  systems.  The  Agent  consists 
of  the  Command  Language  Processor  and  a Tutor. 

Command  Language  Processor  (CLP).  This  serves  as  the  interpreter  for  user 
commands  and  provides  input  editing  functions  and  screen  control.  To  support 
the  neophyte,  the  CLP  provides  spelling  completion  and  correction  for  commands 
and  arguments.  If  the  user  has  any  questions  about  a command  or  its 
arguments,  a prompt  key  shows  the  user  what  the  CLP  has  interpreted  so  far 
about  a command  and  what  it  yet  has  to  resolve. 

Tutor.  This  provides  help  to  on-line  users  by  explaining  commands, 
reporting  errors,  and  providing  tutorial,  exercise,  and  reference 
documentation.  On-line  lesson  material  is  provided  as  an  aid  in  learning  to 
use  the  service.  At  any  time,  the  user  may  enter  this  tutorial  for  whatever 
lesson  he  wishes.  Imbedded  within  the  lessons  are  exercises  which  guide  the 
user  through  execution  of  system  commands  to  illustrate  their  functional 
performance.  Exercises  operate  on  a prestored,  protected  data  base,  so  the 
user  cannot  damage  real  data  by  mistakenly  executing  the  wrong  instruction  to 
the  system. 

Functional  System 

The  functional  aspects  of  the  message  service  are  provided  by  three 
modules  within  each  user  job  (the  Functional  Module  (FM) , the  Message  Access 
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Module,  and  the  File  Access  Module)  and  by  seven  free-running  processes, 
called  daemons,  that  service  all  the  user  jobs  (Coordination,  Message,  File, 
Citation,  Reception,  Archive,  and  Hardcopy). 

The  FM  holds  working  models  of  the  objects  that  the  user  has  open. 
Typically,  this  is  a file,  a message,  and  perhaps  a text  object.  The  FM  sends 
data  to  the  terminal  in  order  to  display  these  objects.  The  FM  is  driven  by  a 

series  of  instructions  passed  to  it  by  the  CLP  whenever  the  user  executes  a 

command  or  depresses  a function  key.  In  addition,  as  editing  changes  are  made 
by  the  user  to  the  objects  being  displayed,  they  are  passed  from  the  terminal 
directly  to  the  FM.  Before  executing  any  command  steps,  the  FM  updates  its 
model  of  the  state  of  the  open  object  to  conform  ti  the  terminal's  model.  The 

message  access  module  is  a separate  process  which  holds  the  actual  file 

representation  of  a message.  When  the  FM  needs  data  about  a message  to  build 
or  change  its  working  model,  it  interacts  with  the  Message  Access  Module.  The 
File  Access  Module  performs  a similar  function  for  user  files. 

Daemons 


Some  user  instructions  cause  the  FM  to  send  requests  to  one  or  more  of  the 
daemons.  The  daemons  are  background  processes  shared  by  all  the  user  jobs. 
They  are  driven  by  requests  put  into  their  input  queues  by  user  jobs  or  other 
daemons . 

The  message  daemon  is  the  only  process  allowed  to  actually  change  the 
message  data  base.  Changes  that  user  jobs  want  made  to  messages  are  sent  to 
the  message  daemon  in  the  form  of  change  files.  The  message  daemon  updates  the 
message  accordingly.  This  centralized  message  daemon  resolves  conflicting 
user  requests.  The  message  daemon  processes  commands  to  coordinate,  chop,  and 
release  messages  generating  appropriate  citations  as  needed. 

The  file  daemon  performs  a function  similar  to  that  of  the  message  daemon 
for  users'  centrally-located  SIGMA  files. 

The  sole  function  of  the  citation  daemon  is  to  build  entries  for  user 
files.  It  takes  as  requests  citations  which  contain  the  information  needed 
for  the  entry. 

The  reception  daemon  takes  incoming  LDMX  traffic,  builds  SIGMA  messages 
from  it,  and  sends  citation  requests  through  the  citation  daemon  to  the 
appropriate  user's  pending  files. 

The  archive  daemon  provides  the  interface  between  the  user  jobs  and  the 
TENEX  archive  system.  It  builds  retrieval  requests  when  a user  asks  for  a 
message  which  has  been  archived.  When  that  message  is  read  from  tape  by  the 
operator,  the  archive  daemon  restores  it  to  the  SIGMA  message  data  base  and 
sends  a retrieval  citation  to  the  user,  telling  him  the  message  requested  is 
back. 

The  hardcopy  daemon  processes  user  print  requests.  This  allows  the  user 
job  to  continue  while  the  printing  is  done  in  the  background. 
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Message  System 


Message  handling  may  be  divided  conveniently  into  two  stages:  message 
preparation  and  message  delivery.  The  former  includes  the  creation  of  the 
draft  message  and  the  coordination  of  this  draft  with  other  users  until  it  is 
signed  off  for  release.  SIGMA  provides  special  editing  facilities  which 
understand  message  formats  and  check  that  the  contents  of  various  fields  are 
legitimate. 

An  originator  drafts  a message,  adds  comments  associated  with  fields  of 
the  message  as  desired,  fills  in  the  CHOP  field  with  user  names,  and  then 
sends  it  for  coordination.  After  coordination  is  complete,  the  message  can  be 
released  by  any  user  who  has  been  given  release  authority  on  the  MME  by  the 
Command.  If  the  released  message  is  an  AUTODIN  message,  then  it  is 
automatically  transmitted  to  the  LDMX.  If  it  is  a memo  or  an  informal  note 
for  another  user  on  the  MME,  then  it  is  delivered  to  the  user's  pending  file. 

The  delivery  stage  involves  delivering  the  message  to  its  ultimate 
recipients,  archiving  it,  and  providing  aids  for  the  user  to  sort  his 
messages,  scan  them,  and  file  them  for  later  retrieval.  The  first  step  in 
this  process  is  to  determine  distribution  for  the  message.  Because  all  formal 
traffic  flows  between  commanders  of  organizations,  it  is  necessary  for  the 
message  system  to  aid  in  determining  the  individual  within  the  command  who 
should  receive  the  incoming  message. 

An  incoming  message  is  delivered  to  an  action  officer's  pending  file  (his 
electronic  in-basket).  To  see  what  new  messages  have  arrived,  a user  asks  to 
DISPLAY  his  pending  file.  He  is  presented  with  an  index  to  the  contents  of 
the  file  with  an  entry  for  each  message.  Each  entry  contains  enough 
information  for  the  user  to  be  able  to  recognize  the  message,  and  the  entry 
serves  as  a reasonably  rich  context  for  subsequent  retrieval;  e.g.,  the 
security  classification  of  the  message,  its  DTG,  its  ORIGINATOR,  subject, 
etc.  For  convenience,  a user  may  attach  comments  to  a file  entry. 

An  action  officer  is  assigned  for  each  incoming  message.  He  may  further 
delegate  the  action  to  a subordinate,  "sell  the  action"  to  some  other  more 
appropriate  officer,  or  act  on  it  himself.  The  details  of  how  this  is  done  is 
described  in  the  previous  sections;  the  MME  system  provides  aids  that  are  used 
by  the  action  officers  and  by  J301.  It  must  keep  track  of  the  action 
assignment  until  such  time  as  the  action  has  been  completed.  On  the  same 
message,  "Information"  copies  may  be  distributed  to  other  action  officers. 
Readboards  for  J3  may  be  built  using  the  messages. 

A user  may  put  messages  into  personal  files  for  later  retrieval.  This 
provides  the  electronic  analog  of  file  cabinets.  Because  the  message  service 
can  retrieve  messages  rapidly,  these  user  files  actually  store  only  citations 
to  messages,  rather  than  the  messages  themselves,  thus  eliminating  multiple 
copies  and  reducing  the  required  computer  storage. 

SIGMA  assists  in  the  distribution  function  by  providing  the  necessary 
functions  as  quick  and  easy-to-use  system  commands.  Several  commands  are 
provided.  ACTION  is  used  to  assign  the  responsibility  for  the  message  and 
marks  the  message  accordingly;  it  causes  a citation  to  the  message  to  be 
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delivered  Co  Che  acCion  assignee  informing  him  of  his  responsibilicy . In 
addicion,  a record  of  Che  assignmenC  is  made  in  a special  AcCion  Log.  In  a 
similar  manner,  FORWARD  is  used  Co  disCribuCe  informacion  copies.  The  names 
of  ACTION  and  INFO  recipienCs  are  appended  Co  appropriaCe  fields  in  Che 
message  icself.  SIGMA  also  allows  users  Co  combine  mulciple  ACTION,  FORWARD, 
and  FILE  operaCions  wich  a single  ROUTE  command. 

To  assisC  Che  user  in  finding  messages  of  inceresc,  SIGMA  provides  Che 
user  wich  func cions  Co  idenCify  and  reCrieve  messages  of  inCeresC.  The  user 
can  enCer  his  reCrieval  criCeria  inCo  whaC  is  called  a selecCor.  The  criceria 
may  include  ORIGINATOR,  a parCieular  sCring  of  CexC  in  Che  subjecc, 

PRECEDENCE,  classif icaCion,  eCc.  These  criceria  may  be  used  Co  selecC  a 
subseC  of  a file  for  sCorage  inCo  a new  file;  ocher  selecCors  may  Chen  be 
applied  Co  Che  new  file  Co  furCher  resCricC  Che  messages  of  inceresc;  in  a 
like  manner,  ocher  messages  may  be  selecced  and  added  Co  Che  file.  The  user 
may  score  Che  selecCor,  specifying  Che  criceria,  wich  an  arbicrary  name  for 
laCer  use.  SIGMA  also  allows  keywords  Co  be  added  Co  received  messages,  which 
may  be  used  for  laCer  reCrieval  of  Che  messages. 

SIGMA  divides  Che  screen  inCo  Chree  areas  (or  windows).  The  Cop  Chree 
lines  of  Che  firsC  area  Cell  Che  user  Che  sCaCe  of  Che  sysCem,  including  such 
informacion  as  Che  name  of  Che  objeccs  Che  user  is  working  wich,  who  is  logged 
on  ac  Che  Cerninal,  Che  cime  of  day,  and  Che  sCaCus  of  Che  insCrucCion 
enCered.  The  lasC  line  is  Che  insCrucCion  window  where  he  encers  commands  Co 
Che  sysCem.  The  Cwo  windows  below  chaC  are  Che  general  working  space. 

Messages  and  CexC  objecCs  are  normally  displayed  Chere.  To  che  righc  of  che 
screen  are  Che  securiCy  lighCs  ChaC  indicaCe  Che  highesc  clasaificacion  of  Che 
daca  displayed. 

The  Cerminal  has  a sCandard  CypewriCer  keyboard,  a sec  of  cerminal  conCrol 
keys,  and  a sec  of  applicaCion-orienCed  funcCion  keys.  An  overlay  labels  Che 
funcCion  keys.  Four  lighCs  in  che  cenCer  of  Che  funcCion  key  area  indicaCe 
Che  securiCy  level  of  Che  informacion  in  Che  window  poinCed  Co  by  che  cursor. 

PERSONNEL  INVOLVED  WITH  MESSAGE  HANDLING 

Personnel  involved  wich  auComaCed  message  processing  in  Che  MME  may  be 
grouped  as  follows: 

Chose  associaCed  wich  che  auComaCed  sysCem  which  CransporCs  messages  (for 

example,  sysCem  operacors  and  Telecommunicacions  CenCer  personnel), 

Chose  associaCed  wich  Che  disCribuCion  and  use  of  individual  messages. 

AuComaCed  sysCem  supporC  includes  Che  conCrol  and  manipulaCion  of  messages 
Chrough  che  elecCronic  conduiC.  The  CommunicaCions  Wacch  Officer  (CWO) 
supervises  Che  operaCions  of  Che  LDMX  compuCer  sysCem.  In  Chis  role,  he 
monicors  operacion  of  Che  LDMX-MME  sysCem  communicaCions  link.  Ic  is  Che 
CWO's  responsibilicy  Co  aid  in  Che  assurance  ChaC  all  required  messages  have 
been  delivered  from  Che  LDMX  Co  SIGMA.  If  any  problems  arise,  Che  CWO  will 
coordinaCe  correccive  efforcs  ac  che  LDMX.  The  only  disCribuCion  and  rouCing 


functions  utilized  at  the  Telecommunications  Center  are  those  automated 
routing  functions  that  send  the  proper  messages  over  the  communications  link.. 

MME  System  Operators  ensure  that  the  message  processing  system  is 
operational  and  that  all  messages  from  the  LDMX  are  accounted  for.  This  is 
done  by  monitoring  the  system  status  and  the  system-generated  LDMX  logs. 

During  the  Limited  Experimental  Use  phase,  the  system  was  operated  on  a 
24-hour  basis  primarily  as  a collateral  duty  of  the  WWMCCS  Operators. 

J301  is  involved  with  distribution  of  messages  to  various  J3  component 
units.  This  office  is  responsible  for  distribution  processing  in  both  the 
paper  and  automated  systems.  Using  the  automated  system  and  a series  of 
selectors,  a terminal  operator  is  capable  of  completing  action  assignments  and 
distribution  of  information  messages  in  a shorter  period  of  time  than  an 
individual  operating  in  the  manual  systems.  Courier  functions  associated  with 
the  manual  system  required  to  transport  bulk  copies  of  individual  messages 
from  the  Telecommunications  Center  to  J301  to  the  component  offices  for 
traffic  handled  by  the  MME  system  a?e  used  now  for  the  parallel  paper  system. 

In  addition  to  these  two  groups,  the  experiment  has  introduced  personnel 
associated  with  the  introduction,  observation,  and  modification  of  the 
automated  process  including  several  elements  of  the  MME  Staff.  These  people 
are  unique  to  the  experiment  environment  and  not  all  would  be  a part  of  an 
operational  message  processing  system. 

Users 


The  user  community  contains  J3  division  administrative  personnel,  action 
officers,  and  watch  standers.  Generally,  use  of  the  system  by  this  group  may 
be  characterized  by  the  following  observations: 

forty-four  of  the  ninety  action  and  watch  officers  have  used  the  system 

over  the  past  five  months.  Characteristics  of  their  activity  include: 

reviewing  the  incoming  traffic  routed  by  J301,  filing,  deleting,  or 
forwarding  as  required; 

reviewing  the  datefile  for  pertinent  messages  (some  officers  have 
created  highly  sophisticated  selectors  with  which  they  can  reduce  the 
datefile  to  a handful  of  relevant  messages  in  a manner  similar  to 
that  used  in  J301). 

As  mentioned  previously,  the  discretionary  access  controls  have  not  been 
implemented;  when  they  are  implemented  in  SIGMA  Release  2.3,  access  to 
messages  found  via  the  datefile  will  be  possible  only  for  the  users  who  have 
proper  access. 

Fifteen  of  the  sixty-one  clerks  assigned  to  J3  offices  have  used  the 
system.  The  relocation  of  offices  outside  the  blockhouse  has  caused  two  to  be 
without  terminals  for  the  last  three  months.  In  general,  use  of  the  system  by 
clerks  has  been  supplemental  to  their  use  of  the  paper  system.  Retrieving  and 
printing  messages  have  been  the  primary  features  of  the  clerks'  use  of  the 
system.  Some  use  has  also  been  made  of  the  informal  note  capability  by  the 
clerks. 
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SECTION  5 


1 


DATA  COLLECTION 

The  MME  is  designed  Co  evaluate  the  utility,  design,  and  organizational 
impact  of  a computer-aided  message  system  in  a military  environment.  This 
section  describes  the  data  collection  techniques  being  used  to  collect  the 
data  necessary  for  this  evaluation. 

METHODS 

Before  SIGMA  was  operational  at  CINCPAC,  baseline  measures  were  collected 
on  manual  message  handling  activities.  Questionnaires,  checksheets,  and 
observation  were  the  primary  methods  used  to  collect  these  data.  The  results 
of  this  data  collection  effort  are  outlined  in  section  3. 

Data  on  automated  message  handling  activities  (use  of  SIGMA  by  CINCPAC 
personnel)  are  being  collected  in  two  different  ways.  First,  data  on  user 
sessions  are  being  collected  by  the  message  system  itself.  Additional  data 
will  be  obtained  from  interviews  with  system  users  and  observations  of  system 
use  made  by  MME  personnel  and  will  be  used  to  augment  the  data  collected 
on-line. 

The  data  being  collected  by  the  message  system  are  sent  to  MITRE,  Bedford, 
for  analysis.  In  order  to  have  a quicker  look  at  system  statistics, 
additional  data  are  collected  and  analyzed  manually  by  the  on-site  team.  The 
data  analyzed  by  MITRE  for  this  report  cover  the  period  of  20  July  1978,  to  27 
September  1978.  The  KA-10  processor  was  replaced  by  the  more  powerful  KL-10 
processor  on  16  October  1978.  Some  of  the  manually  analyzed  data  will  be  used 
in  this  report  to  identify  early  trends  after  the  installation  of  the  KL-10. 
Where  appropriate,  the  source  of  data  is  indicated. 

Automated 


Data  collected  by  the  message  system  are  collected  primarily  from  two 
different  on-line  sources  - the  detailed  data  collection  files  (the  dcf)  and 
session  transcripts.  Every  two  weeks,  a computer  tape  containing  the  dcf  and 
session  transcripts  for  that  two-week  period  is  sent  to  MITRE  from  CINCPAC  for 
analysis.  The  dcf  are  the  major  source  of  data;  they  consist  of  coded 
representations  of  each  user's  activity  and  are  designed  for  computer 
analysis.  They  identify  every  kind  of  instruction  entered  into  SIGMA 
(including  the  five  break  keys,  "prompt",  "help",  "execute",  "expand",  and 
"cancel")  and  the  resulting  system  feedback,  such  as  error  messages.  For  each 
instruction,  the  dcf  include  timing  data  (real  (clock))  time,  system 
processing  time,  and  cpu  time)  and  a unique  identifier  for  each  of  the  objects 
(file,  message,  selector,  text  object,  etc.)  being  acted  upon.  For  errors  and 
break  key  hits,  the  dcf  include  some  timing  information  and  identifiers. 

To  reduce  all  this  information  into  data  which  are  meaningful  for  a 
comparison  of  manual  and  automated  message  handling,  data  reduction  algorithms 
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were  developed  for  each  of  the  ten  task-oriented  areas  (message  distribution, 
message  usage,  filing,  message  retrieval,  creation/drafting,  coordination, 
release,  transmission  and  post-transmission,  crises  and  exercises,  and  message 
system  statistics).  Each  of  these  functional  areas  has  its  own  set  of  data 
reduction  algorithms  and  variables,  preliminary  versions  of  which  are 
available  in  the  test  procedures.  These  algorithms  are  the  basis  for  the  data 
reduction  programs  and,  thus  are  constantly  being  revised  as  changes  are  made 
both  to  the  format  of  the  dcf  and  to  the  scope  of  the  experiment. 
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In  addition  to  the  task-oriented  statistics  which  are  collected  for  each 
functional  area,  the  dcf  provide  raw  data  for  two  other  types  of  analysis: 
system  usage,  and  system  load  and  performance.  Data  on  system  usage  identify 
the  ways  in  which  SIGMA  is  being  used  by  J3  personnel  to  accomplish  their 
daily  message-handling  tasks.  This  includes  the  amount  of  time  a user  spends 
on-line  during  a given  day,  the  types  of  instructions  he  executes  while 
on-line,  and  how  difficult  he  finds  the  system  to  use  (e.g.,  how  many  errors 
he  makes,  how  often  he  needs  to  take  advantage  of  the  on-line  help  facilities). 


System  load  is  a measure  of  both  the  number  of  users  logged  on  at  a given 
time  and  of  each  user's  level  of  activity;  system  performance  is  a measure  of 
the  system  response  time  to  useT-entered  instructions.  Data  on  system  load 
and  performance  represent  an  additional  data  reduction  capability  which  is  not 
an  integral  part  of  the  objectives  of  the  experiment.  Response  time  is 
included  with  the  other  data  reduction  capabilities  because  system  response 
influences  how  (or  whether)  a user  uses  the  system.  For  example,  a user  might 
avoid  using  the  system  during  periods  of  heavy  load  because  response  time 

would  be  too  slow.  Data  on  response  time  will  show  periods  when  the  system  is 

unusually  slow. 

Session  transcripts,  the  second  on-line  source  of  data,  record  the 
expanded  instruction  entered  by  the  user  and  the  system  feedback  (such  as 
errors)  in  text  form.  The  associated  time  for  each  instruction  or  feedback  is 

included.  Session  transcripts  record  user  sessions  in  a format  which  is 

human-readable,  making  them  ideal  for  manual  analysis.  The  data  obtained  from 
manual  analysis  of  session  transcripts  will  be  used  only  occasionally  to 
supplement  the  data  obtained  from  the  dcf.  The  major  advantage  of  the  session 
transcripts  is  that  patterns  of  system  usage  which  would  be  difficult  to 
detect  with  computer  analysis  can  be  detected  by  human  analysis. 


Manual 

While  the  data  collected  on-line  will  demonstrate  how  the  system  is  being 
used,  they  will  not  show  what  the  user  is  trying  to  accomplish  while  using 
SIGMA.  Thus,  there  is  a need  for  two  types  of  off-line  data  collection  - 
interviews  and  observations. 


Interviews  will  be  used  to  determine  how  the  user  accomplishes  various 
tasks  on-line.  They  will  also  be  used  to  collect  suggestions  for  improvements 
or  changes  to  SIGMA  and  to  collect  information  about  the  users’  perceptions  of 
the  system's  effect  on  their  overall  message  handling  activities.  A proposed 
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outline  calls  for  the  following  subjects  to  be  covered  in  each  interview:  the 
purpose  of  the  interview,  the  user’s  description  of  a typical  SIGMA  session,  a 
comparison  of  message  handling  with  SIGMA  and  message  handling  with  the  paper 
system  (faster,  slower,  easier,  harder),  comments  on  SIGMA  features  (which  are 
the  most  useful,  which  need  modificaton,  what  new  features  the  user  feels 
should  be  implemented),  and  connents  on  the  user  interface  (items  such  as  the 
on-line  help  facilities,  function  keys,  editing  capabilities,  view  and  display 
windows,  etc.).  Special  questions  relating  to  the  role  or  type  of  the  user 
being  interviewed  will  also  be  asked.  During  the  interviews,  all  these  points 
will  be  covered  and  the  answers  categorized  for  future  comparisons. 

While  some  practice  interviews  have  already  taken  place  and  a preliminary 
interview  conducted  for  this  report  (see  section  8),  the  official  interview 
schedule  depends  on  the  installation  of  different  versions  of  SIGMA  and  on 
increased  user  experience.  Not  all  users  will  be  interviewed  each  time. 

Users  will  be  selected  on  the  basis  of  the  amount  of  system  use  (including 
light  as  well  as  heavy  users)  and  type  of  use.  All  users  will  be  interviewed 
at  least  once  and  at  most  twice  during  the  course  of  the  experiment. 

There  are  no  arrangements  for  formal  observation  of  automated  message 
handling  activities  during  the  experiment.  Rather,  when  questions  arise  from 
analysis  of  the  dcf  or  session  transcripts,  MME  personnel  will  be  asked  to 
clarify  or  explain  these  problems,  based  on  their  informal  observations  of 
CINCPAC  SIGMA  use.  In  addition,  to  augment  the  list  of  suggestions  for 
changes  and  improvements  to  SIGMA  obtained  in  the  interviews,  a log  is  being 
kept  of  suggestions  made  spontaneously  during  informal  interactions  between 
MME  personnel  and  CINCPAC  SIGMA  users. 


PERIODS  OF  DATA  COLLECTION 

The  data  for  this  section  of  the  report  were  collected  at  CINCPAC  during 
the  period  between  20  July  1978  and  27  September  1978  with  the  exception  of  a 
few  days  when  either  a)  the  data  collection  facilities  had  not  been  enabled 
after  testing,  or  b)  there  were  problems  with  SIGMA  that  day  that  made  it 
impossible  for  the  system  to  be  used. 
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SECTION  6 
TRAINING 

Two  MME  objectives  are  a)  to  develop  and  document  a process  for  the 
installation  of  a military  message  system  into  an  operatonal  command,  and  b) 
to  determine  the  nature  of  training  requirements  associated  with  the 
implementation  of  a military  message  system.  The  training  experience  is 
reported  in  reference  (k);  the  following  is  a summary  supplied  by  Dave  Miller 
of  MITRE. 

The  training  program  designed  for  MME  users  has  four  main  components:  an 
introductory  lecture,  on-line  lessons  and  exercises,  printed  documentation, 
and  one-on-one  training  sessions. 

INTRODUCTORY  LECTURE 

The  introductory  lecture  is  the  first  contact  that  most  users  have  with 
the  MME.  It  is  an  opportunity  to  indoctrinate  users  by  telling  them  a little 
about  the  background  of  the  experiment,  discussing  and  demonstrating  some  of 
the  features  of  the  system,  and  bringing  users  to  the  level  at  which  they  can 
log  on  and  start  taking  lessons.  The  lecture  has  four  parts:  introduction 
and  goals,  the  SIGMA  system,  the  terminal,  and  help  for  the  users. 

The  lectures  are  given  to  groups  of  two  to  seven  people,  with  five  being  a 
typical  size.  It  is  given  using  an  MME  terminal  which  contains  special 
electronics  that  drive  a large-screen  monitor  for  ersier  visibility.  The 
lecture  is  about  an  hour  and  a half;  at  the  end,  the  attendees  are  given 
accounts  on  SIGMA  and  a handout  with  information  on  logging  on  and  taking 
lessons. 

The  lecture  seems  to  have  been  fairly  well  received.  Reactions,  of 
course,  have  varied  widely  among  the  recipients.  A few  users  asked  numerous 
questions,  but  most  seem  to  accept  what  was  told  them  without  question.  The 
amount  of  material  covered  by  the  lecture  has  grown  smaller  as  we  continue  to 
indoctrinate  new  users;  the  number  of  major  sections  has  been  reduced  from  six 
to  four. 


LESSONS  AND  EXERCISES 

The  on-line  lessons  and  exercises  are  intended  for  use  at  the  terminal; 
they  are  expository  in  nature  and  discuss  aspects  of  commonly  used 
instructions.  Exercises  allow  users  to  practice  the  instructions  that  are 
being  taught  in  the  lessons.  Lessons  currently  available  in  SIGMA  include: 


Lesson  1 : 
Lesson  2A: 
Lesson  2B: 
Lesson  3A: 
Lesson  3B: 


A General  Description  of  the  SIGMA  Service 
Beginning  to  Use  the  SIGMA  Service 
Beginning  to  Use  the  SIGMA  Service 
The  Filing  Service-Basic  Filing  Techniques 
Advanced  Filing  Techniques 
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Lesson  3C: 
Lesson  4: 
Lesson  5A: 
Lesson  5B: 
Lesson  5C: 
lesson  6: 
Lesson  7: 


Special  Files 

Message  Reception  and  Distribution 
Text  Objects 

Editing  Instructions  for  Text  Objects 
Editing  Instruction  Keys 
Message  Drafting 
Message  Review  and  Coordination 


DOCUMENTATION 


Reference  Manual 

The  SIGMA  Reference  Manual  is  a compilation  of  detailed  information  about 
SIGMA.  It  serves  more  as  a place  to  find  out  the  details  of  instructions  one 
wants  to  carry  out  rather  than  as  a place  to  find  out  which  instructions  to 
use. 

Primer 


ISI  created  a Primer  for  use  by  those  not  needing,  or  ready  for,  the 
detail  of  the  Reference  Manual.  It  is  written  in  a conversational  style  and 
is  easier  to  read  for  those  who  don't  know  the  system  well.  The  topics  in  the 
Primer  cover  most,  but  not  all,  of  the  SIGMA  system. 

On-Site  Handouts 


From  time  to  time,  special  written  material  has  been  generated  on-site  and 
distributed  to  some  or  all  of  the  users.  One  example  of  this  is  a discussion 
concerning  taking  lessons  on  SIGMA.  This  handout  was  produced  because  of  the 
difficulties  some  of  the  early  trainees  were  having  remembering  all  the  things 
they  were  expected  to  do  in  order  to  log  on  and  start  taking  lessons. 

One-on-One  Training 

The  fourth  prong  of  the  training  approach  involves  supplying  users  with 
individual  training  sessions  as  required.  These  sessions  can  be  general  in 
nature  dealing  with  all  aspects  of  the  system,  or  tailored  to  fit  the  needs  of 
a particular  user's  job.  They  may  consist  only  of  the  training  representative 
watching  the  user  performing  his  message-handling  duties  and  making  occasional 
suggestions,  or  they  may  involve  some  study  of  the  office  functions  by  the 
training  representative,  followed  by  distribution  of  handouts  and  follow-up 
sessions . 


TRAINING  EXPERIENCE  SUMMARY 
Preliminary  Period 

During  the  preliminary  shakedown  and  learning  period  from  May  1977  to  July 
1978,  the  system  was  used  primarily  by  project  personnel  for  testing  and 
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debugging  purposes.  Early  in  this  period,  a few  "friendly  users"  were  trained 
- people  who  were  designated  by  J3  as  points  of  contact,  or  who  were  expected 
to  be  heavy  users.  This  included  several  key  clerical  personnel,  both  in  the 
operating  divisions  of  J3  and  in  J301.  These  users  were  sympathetic  to  the 
system's  growing  pains.  Table  2 summarizes  the  training  during  the 
preliminary  period. 


Table  2.  SIGMA  Training  During  Preliminary  Period 


Description  of  Training 


Number  of  Users 


J3 


Given  Introductory  Lecture 
From  Other  Directorates 
People  Given  Lecture 
From  Non-Test  Divisions 
Test  People  Given  Lecture 
Scheduled  to  Leave 
Potential  J3  Training  Population 


J3 


97 

13 

84 

18 

66 

15 

51 


Because  of  some  problems  in  collecting  data,  the  statistics  in  Table  3 
understate  to  some  extent  the  number  of  lessons  taken. 


Table  3.  J3  Lesson-Taking  During  the  Preliminary  Period 


Lesson 

Number 

Number  i 

At  Least  One 

29 

Lesson 

1 

25 

Lesson 

2A 

24 

Lesson 

2B 

10 

Lesson 

3A 

15 

Lesson 

3B 

9 

Lesson 

3C 

7 

Lesson 

4 

8 

Lesson 

5A 

6 

Lesson 

5B 

5 

Lesson 

5C 

5 

Lesson 

6 

7 

Lesson 

7 

6 
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In  Table  3,  some  lesson-taking  statistics  are  reported.  The  lesson-taking 
occurred  during  the  preliminary  period,  and  was  done  by  the  66  J3  users 
discussed  above.  Multiple  takes  of  a lesson  are  not  included. 

The  following  are  a few  factors  that  explain  why  the  numbers  are  low. 
Approximately  15  of  the  66  users  were  scheduled  for  transfer  away  from 
CINCPAC.  They  may  have  been  less  motivated  than  most  to  take  lessons,  in  the 
belief  that  they  would  never  use  the  system.  Of  the  remaining  51,  six  were 
officers  of  the  0—6  level  (Colonel  or  Navy  Captain),  who  seldom  do  message 
handling  themselves.  Only  one  showed  any  lesson  activity.  Also,  in  some 
offices  one  person  seems  to  have  become  the  one  responsible  for  running  the 
terminal,  and  others  have  taken  little  interest  in  learning  SIGMA. 

Limited  Use 

During  the  limited-use  period,  one-on-one  sessions  have  been  held  with 
several  of  the  users.  In  particular,  from  the  10  May  1978  until  28  June  1978, 
the  trainer  conducted  eleven  J301  message  routing  sessions  for  periods  of  time 
ranging  from  one  to  three  hours.  Because  message  routing  is  an  important  J301 
function,  this  was  emphasized  in  the  training. 

Acceptance  of  the  on-line  lessons  varies  widely  (as  is  attested  to  by  the 
lesson-taking  statistics).  A few  users  were  quite  impressed  with  the  novel 
(to  them)  approach  of  the  on-line  lessons.  Two  lieutenant  colonels  who  are 
very  proficient  users  feel  that  the  lessons  help  somewhat,  but  they  believe 
that  the  novice  user  must  be  willing  to  dig  things  out  for  himself,  if  he 
hopes  to  become  better.  One  of  these  same  officers  said  that  he  found  the 
SIGMA  on-line  prompt  and  help  features  more  useful  than  the  lessons.  An 
examination  of  the  lesson-taking  statistics,  coupled  with  personal  obser- 
vation, suggests  that  many  people  found  the  early  lessons  helpful,  but  didn’t 
continue  with  them  beyond  the  bare  minimum  they  needed  to  do  their  jobs  on  the 
terminal. 


TRAINING  LESSONS  LEARNED  SO  FAR 
On-Line  Lesson  Not  Enough 

Although  the  on-line  lessons  have  proved  to  be  very  useful,  they  have  not 
been  accepted  to  the  degree  hoped.  Statistics  presented  in  the  section  on 
training  experience  show  that  the  lower-numbered  lessons  received  more  use 
than  the  higher-numbered  ones.  This  suggests  that  either  the  users  tired 
after  taking  a few  lessons,  or  that  they  learned  enough  from  the  early  lessons 
to  do  most  of  their  tasks.  On-site  observation  suggests  that  the  reason  is  a 
combination  of  the  two. 

Much  Personal  Attention  Needed 

This  is  a corollary  of  the  previous  observation.  Several  (but  not  all)  of 
the  users  seem  to  respond  better  to  individual  instruction  than  to  taking  the 
on-line  lessons.  This  seems  to  be  particularly  true  for  enlisted  personnel. 


Primer  a Used  More  Than  Reference  Manuals 


It  has  been  our  observation  that  the  primers  are  used  much  more  than  the 
reference  manuals.  There  are  probably  two  reasons  for  this:  a)  the  primer  is 
written  in  a more  conversational,  breezy  style,  and  b)  at  the  present  stage  of 
the  experiment,  introductory  level  material  is  needed  more  than  the  details  of 
features  and  instructions. 

Evolution  of  Introductory  Lecture 

The  introductory  lecture  has  been  given  almost  thirty  times.  Using 
attendee  reaction  as  a guide,  the  lecture  has  evolved  gradually  into  its 
present  form.  Initially,  the  lecture  was  longer,  having  additional  sections 
on  message  handling  and  message  composition.  It  proved  difficult  to  cover  all 
this  material  in  the  hour  and  a half  or  so  that  was  scheduled  for  a lecture, 
and  gradually  the  last  two  sections  were  dropped.  The  demonstration  seemed  to 
be  the  part  of  the  remaining  material  that  was  best  received,  and  so  gradually 
the  time  devoted  to  it  increased,  while  the  time  spent  on  presenting  slides 
and  discussing  the  background  of  the  experiment  and  fundamentals  of  the  system 
was  proportionately  reduced. 

System  Stability  Important 

Trainees  thrive  in  an  environment  of  stability  and  reliability.  That  is, 
it  is  expecting  a lot  to  ask  a trainee  to  use  a system  which  is  often 
unavailable,  responds  very  slowly,  or  does  not  respond  in  the  way  that  the 
instructional  material  leads  him  to  expect.  Thus  far  during  the  MME,  there 
have  been  periods  of  system  instability,  such  as  system  interruptions  that 
require  the  trainee  to  log  on  again  and  to  do  some  work  to  get  back  to  the 
previous  point  in  his  work.  CINCPAC  users  have  been  doggedly  patient  during 
these  trying  periods,  but  without  a doubt  some  harm  has  been  done  to  the 
training  program  because  of  the  interruptions. 

Testing  as  an  Aid  to  Training 

Using  people  from  the  user  community  for  system  testing,  such  as  the  load 
tests  used  to  measure  performance  in  new  SIGMA  releases,  appears  to  have  had 
an  overall  beneficial  effect  on  training.  For  one  thing,  during  periods  of 
project  delay  due  to  hardware  and  software  difficulties,  it  helps  keep  user 
interest  up.  For  another,  the  scripts  that  these  users  were  asked  to  follow 
during  the  tests  helped  make  them  aware  of  some  system  capabilities  that  they 
may  not  have  fully  appreciated  otherwise. 

Project  Stretch-Out  Problems 

The  limited  use  phase  of  the  project  has  suffered  from  a couple  of  false 
starts  - that  is,  there  were  some  optimistic  estimates  of  when  it  would 
begin.  Since  we  felt  it  was  necessary  to  start  training  people  a few  weeks  in 
advance  of  that  time,  a good  many  people  were  given  the  introductory  lecture 
and  urged  to  commence  their  lesson-taking  as  much  as  five  or  six  months  in 


advance.  Not  all  users  suffered  from  this  delay,  but  certainly  for  some,  the 
momentum  developed  during  lesson-taking  was  dissipated  before  the  limited  use 
phase  began.  It  seems  reasonable  to  conclude  that  the  project  suffered  some 
loss  of  credibility  with  these  users. 

Access  Difficulties 

One  aspect  of  the  MME  environment  that  has  caused  more  problems  than 
anticipated  is  chat  of  lack  of  free  access  to  some  of  the  work  spaces  in  which 
the  terminals  are  installed.  This  comes  about  because  of  the  high  security 
level  at  which  some  of  the  offices  operate.  Although  some  project  personnel 
have  the  necessary  clearances,  others  do  not,  and  on  occasion  someone  not 
completely  familiar  with  a problem  may  be  the  one  who  has  to  respond  to  it. 
Requests  for  advance  clearances  often  require  a long  processing  time. 

JOINT  MME  STAFF  - J3  STAFF  TRAINING  AIDS 

Two  additional  aids  to  training  have  been  prepared  by  J341,  the  J3  branch 
directly  involved  with  introducing  the  MME.  They  are  a set  of  milestones  and 
a user's  guide. 

The  milestones  are  a series  of  short  scenarios,  each  covering  some 
message-handling  function.  The  scenarios  consist  of  detailed  SIGMA 
instructions  that  achieve  that  particular  function,  using  sample  file  names, 
message  numbers,  etc.  Different  milestones  are  presented  to  different  users. 
For  example,  action  officers  have  a set  of  milestones  with  emphasis  on  file 
handling  and  message  reading,  while  senior  officers  have  a set  concentrating 
on  message  coordination  and  release.  By  looking  at  session  transcripts,  J3  is 
able  to  monitor  the  progress  of  the  staff  in  reaching  its  milestones. 

Periodic  reports  are  issued  by  J3  showing  proficiency  ratings  based  on  the 
number  of  milestones  reached,  as  well  as  the  amount  of  time  spent  using  the 
MME . 

The  purpose  of  the  user's  guide  is  to  present  a series  of  very  detailed, 
cookbook-like  instructions  in  the  use  of  the  MME  to  do  specific  tasks.  The 
projected  users  of  the  guide  are  seen  as  those  who  do  not  have  time  to  learn 
to  use  the  MME  by  regular  methods,  but  who  must  be  able  to  carry  out  certain 
narrowly  defined  tasks  from  time  to  time.  Sections  of  the  user's  guide 
concentrate  on  these  specific,  narrow  tasks,  and  are  written  at  a very 
detailed  level.  For  example,  they  include  instructions  on  moving  the  cursor 
from  one  part  of  the  screen  to  another  before  the  next  operation,  which  is  the 
sort  of  knowledge  most  other  users  are  presumed  to  have  learned  earlier. 
Normally  this  level  of  detail  is  omitted  from  further  training.  At  the  time 
this  report  is  being  written,  the  user's  guide  is  not  complete,  but  sections 
are  being  added  to  it  at  the  rate  of  about  one  a week. 
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SECTION  7 

PATTERNS  OF  SYSTEM  USE 


Limited  Experimental  Use  (LEU)  of  the  computer-aided  message  handling 
system,  SIGMA,  began  in  July  1978.  During  this  period  selected  members  of  the 
Operations  Directorate  (J3)  have  been  directed  to  use  SIGMA  for  certain  tasks, 
as  described  elsewhere  in  this  report.  The  manual  system  has  remained  the 
primary  system  during  LEU. 

Data  describing  SIGMA  usage  during  the  intial  period  of  LEU,  from  late 
July  through  late  September,  have  been  collected,  reduced,  and  analyzed.  The 
analysis  is  reported  in  reference  (1);  the  information  in  this  section  has 
been  extracted  from  reference  (1).  The  results  are  reported  in  both  a 
quantitative  and  qualitative  manner.  The  description  of  the  quantitative  use 
of  SIGMA  begins  on  the  following  page  and  deals  with  the  amount  of  use  SIGMA 
has  had  in  terms  of  time  users  spent  logged  on  and  the  numbers  of  commands 
executed.  The  last  part  of  this  section  deals  with  the  type  of  use  SIGMA  has 
had  in  terms  of  the  commands  and  functions  executed  by  different  kinds  of 
users . 

In  early  September  a new  version  of  SIGMA  was  installed  at  CINCPAC.  This 
version  included  some  enhanced  functions  and  changes  to  improve  performance. 
Therefore,  the  data  from  July  and  August  may  differ  from  September  data  not 
only  as  a result  of  increased  user  familiarity  with  the  system,  but  also  as  a 
result  of  the  changes  made  to  the  system  itself.  When  comparing  data  in  this 
report  with  future  reports,  differences  in  system  use  may  be  attributable  to 
changes  in  performance  (because  of  the  addition  of  the  more-powerful  CPU)  as 
well  as  increased  user  experience. 

In  addition,  the  number  and  configuration  of  user  terminals  have  been 
changing.  In  July,  a terminal  for  J32  was  added  making  a total  of  13 
terminals  in  the  J3.  In  August,  J3,  J30,  J30A,  J33,  and  J332  moved  to  Wing 
3B,  reducing  the  number  of  terminals  in  use  to  11.  In  September,  a terminal 
for  J313  was  added  for  a total  of  12.  At  the  end  of  November  there  was  a 
total  of  16  terminals  available  for  the  experiment. 

Throughout  this  section  of  the  report,  users  are  referred  to  as  belonging 
to  different  roles  or  types.  These  are  related  to  the  functional  positions 
the  users  hold  in  the  organization.  Types  include  division/branch 
administrative  (DBADM),  who  are  clerical-level  personnel  with  administrative 
responsibility;  J301,  the  Administrative  Section  of  J3;  action  officers 
(DBACT),  divided  into  three  categories  according  to  volume  of  incoming 
messages;  division/branch  clerical  (DBCL),  whose  duties  are  primarily 
clerical;  members  of  the  command  center  watch  team,  who  are  the  duty  director 
of  operations  (DDO),  air  desk  duty  officer  (AIR),  and  surface  desk  duty 
officer  (SUR);  and,  members  of  J314  and  the  Joint  Reconnaissance  Center  (JRC), 
who  are  grouped  together  because  they  shift  between  the  JRC  and  action  officer 
positions . 


For  all  of  these  users,  and  particularly  for  the  action  officers,  it 
should  be  noted  that  there  is  a great  deal  of  individual  variability.  System 
use  during  LEU  has  been  encouraged  but  not  required.  Some  people  may  choose 
to  use  the  system  as  their  primary  message-handling  tool,  others  may  not.  The 
data  presented  in  this  report  are  therefore  descriptive,  rather  than 
predictive.  They  are  meant  to  show  how  the  system  has  been  used  during  its 
initial  phases  of  implementation  and  to  serve  as  a basis  of  comparison  for 
future  system  use. 


The  data  do  not  yet  permit  one  to  draw  conclusions 
about  system  utility  in  an  operational  environment 


AMOUNT  OF  USE 


The  amount  of  system  use  during  initial  LEU  will  be  reported  from  two 
orientations.  The  amount  of  use  per  hour  shows  the  time  of  day  that  the 
system  is  loaded,  regardless  of  the  identity  of  the  users.  Usage  by  user  type 
shows  the  amount  of  load  placed  on  the  system  by  different  kinds  of  users. 

(The  actual  functions  being  used  by  these  people  are  discussed  in  the  next 
section. ) 


HOURLY  USE 


Hourly  system  use  is  described  by  two  measures:  user-hours  per  hour  and 
commands  per  hour.  User-hours  represents  the  amount  of  time  users  spent 
logged  on  during  an  hour.  It  does  not  necessarily  represent  the  total  number 
of  users  who  were  logged  on  that  hour.  For  example,  a value  of  two  for 
user-hours  might  account  for  two  users  logged  on  throughout  the  hour,  but  it 
might  also  account  for  one  user  logged  on  throughout  the  hour  and  two  users 
logged  on  for  one-half  hour  each.  Furthermore  these  users  might  be  logged  on 
concurrently  or  consecutively. 


Users  may  be  logged  on  but  not  actively  using  the  system.  Thus,  commands- 
per-hour  gives  a somewhat  better  measure  of  system  load.  This  measure  shows 
the  number  of  commands  executed  during  an  hour,  regardless  of  the  number  of 
users.  A value  of  40  could  represent  one  user  executing  40  commands  or 
several  users  executing  a total  of  40  commands. 


Together,  these  two  measures  provide  information  about  the  amount  of  usage 
the  system  receives  throughout  the  day. 


The  mean  number  of  user-hours  per  hour  and  mean  number  commands  per  hour 
were  calculated  for  four  periods  between  20  July  and  27  September  1978. 
Throughout  the  entire  period,  patterns  of  use  were  similar,  so  overall  means 
were  calculated,  and  are  presented  in  Figures  1 and  2.  (Although  patterns  of 
use  were  similar,  the  four  periods  do  show  a trend  for  increased  usage  as  the 
LEU  progressed.  The  detailed  data  are  shown  in  appendix  A.  Also,  see  section 
8 of  this  report.) 
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The  mean  number  of  user-hours,  Fig.  1,  shows  what  one  would  expect  in  a 
staff  environment.  There  is  a sharp  rise  in  the  number  of  user-hours  at  0700, 
a modest  dip  at  1200,  a post-lunch  rise  followed  by  a decreasing  number  of 
users  logged  on  until  1800,  when  the  command  center  watch  teams  change.  There 
is  a low  level  of  activity  overnight. 

The  pattern  of  use  shown  in  the  number  of  commands  executed,  Fig.  2,  is 
quite  similar.  During  the  day,  use  peaks  between  0800  and  0900  and  again 
between  1400  and  1500.  The  relationship  between  users  logged  on  and  users 
actively  using  the  system  is  shown  by  the  contrast  between  the  stable  number 
of  user-hours  during  normal  off-duty  hours  and  the  variable  number  of  commands 
executed  during  these  hours.  Most,  if  not  all,  of  this  off-hour  usage  can  be 
attributed  to  CCWT  and  JRC  watch  officers. 

USE  BY  TYPE 

As  discussed  earlier,  users  have  been  divided  into  several  types,  based  on 
their  position  in  the  organization.  Figures  3 and  4 show  system  use  by  type, 
measured  by  mean  on-line  time  and  mean  number  of  commands  daily.  Taken 
together,  these  measures  show  different  styles  of  use  as  well  as  different 
levels  of  use  by  the  different  user  types. 

Division  and  branch  administrative  personnel  (clerical  personnel  with 
primarily  administrative  duties)  tend  to  log  on  in  the  morning  and  leave  the 
terminal  on  most  of  the  day.  This  is  shown  in  Fig.  3,  where  the  mean  daily 
on-line  time  for  an  administrative  user  is  slightly  over  five  hours.  Actual 
use,  however,  is  quite  low.  In  Fig.  4 we  see  that  an  administrative  user 
executes  an  average  of  fewer  than  20  commands  a day.  Presumably  these  users, 
who  often  need  to  respond  to  requests  from  their  chiefs,  are  keeping  the 
terminals  on  so  they  can  respond  quickly. 

It  is  interesting  to  compare  this  style  of  usage  with  that  of  J301,  the 
administrative  section  responsible  for  distributing  messages  within  the 
directorate  and  for  retrieving  older  messages  on  request.  The  J301  user 
spends  an  average  of  2.25  hours  on-line  daily,  but  executes  an  average  of  80 
commands.  These  users  log  on,  distribute  messages,  and  when  the  job  is 
finished,  log  off.  Their  on-line  time  is  less  than  half  that  of  the  division 
and  branch  administrative  personnel,  but  a J301  user  executes  more  than  four 
times  as  many  commands. 
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For  the  three  categories  of  action  officers,  the  style  is  less  clear,  but 
the  data  suggest  that  they  are  more  likely  to  log  on,  do  a job,  and  log  off, 
than  to  leave  the  terminal  logged  on  when  it  is  not  in  use.  Since  many  of  the 
terminals  are  shared  by  several  action  officers,  terminal  access  may  be  a 
factor  for  these  users.  For  example,  there  are  currently  more  active  users  in 
the  second  category  than  in  the  others,  so  the  shorter  on-line  time  for  a user 
in  this  category  may  simply  represent  the  need  to  share  terminals.  Actual 
use,  as  shown  by  the  mean  number  of  commands  executed,  shows  that  an  action 
officer  in  category  3 presently  executes  more  commands  than  an  action  officer 
in  either  of  the  other  two  categories.  This  is  interesting  because  category  3 
branches  have  the  lightest  incoming  message  load,  and  during  LEU  the  system 
has  been  used  most  for  incoming  messages.  However,  the  computer-aided  system 
has  been  secondary  to  the  manual  system,  so  it  is  possible  that  officers  in 
the  more  heavily  loaded  branches  have  chosen  not  to  use  the  system  as  much  as 
they  might  have  otherwise.  In  any  case,  individual  differences  among  all 
officers  are  more  noticable  than  differences  among  categories  of  officers. 

Personnel  whose  duties  are  primarily  clerical  also  seem  to  log  on,  do 
their  job,  and  log  off.  The  average  on-line  time  for  a clerical  user  was 
slightly  over  two  hours,  while  the  average  number  of  commands  executed  was 
just  over  30. 

In  the  command  center,  the  air  and  surface  desks  each  have  a terminal,  and 
the  DDO  does  not.  Thus  it  is  not  surprising  that  the  usage  by  DDOs  has  so  far 
been  very  light.  The  air  and  surface  desk  duty  officers  and  the  JRC  watch 
officer  (who  also  has  his  own  terminal)  all  tend  to  leave  their  terminals 
logged  on  for  long  periods.  An  air  desk  duty  officer  averages  about  seven 
hours  of  on-line  time  daily,  while  the  surface  desk  and  JRC  officers  average 
about  six  hours  on  line.  (Note  that  these  officers  serve  on  12-hour  shifts.) 
Of  these  officers  the  surface  desk  officer  is  the  most  active,  with  an  average 
close  to  120  commands  daily.  The  air  and  JRL  duty  officers  average  around  70 
commands  daily,  about  the  same  as  a busy  action  officer. 

For  some  types  of  users  the  daily  load  on  the  system  for  that  type  as  a 
group  can  be  determined  by  multiplying  the  average  time  or  number  of  commands 
by  the  number  of  users;  for  other  types  of  users  this  is  not  true.  The 
difference  is  that  some  types  of  users  have  specific,  finite  jobs  which  must 
be  accomplished,  while  tasks  for  other  types  of  users  are  more  open-ended. 

For  example,  in  J301  use  is  primarily  dedicated  to  distributing  messages.  It 
does  not  matter  if  one  or  two  people  in  J301  use  the  system  for  this  task; 
each  message  has  to  be  distributed  once  (generally)  and  when  all  have  been 
distributed,  the  task  is  complete.  On  the  other  hand,  each  action  officer  has 
to  review  and  read  messages  in  his  own  in-box,  regardless  of  what  other  action 
officers  have  done.  Thus  adding  action  officers  to  the  system  adds  much  more 
load  than  adding  J301  users.  In  the  command  center  there  is  one  duty  officer 
per  desk  per  shift.  Thus  the  daily  load  for  users  of  each  of  those  types, 
combined,  would  be  about  twice  that  of  an  individual  CCWT  duty  officer. 

Figures  5 and  6 show  the  average  daily  load  for  all  users  in  a type.  Note 
that  in  terms  of  on-line  time  (Fig.  5),  modest  increases  in  time  are  shown  for 
J301,  division/branch  clerical,  and  DDO  types.  Moderate  increases  are  shown 
for  division/branch  administrative  and  category  1 and  3 action  officers. 

Large  increases  are  shown  for  category  2 action  officers,  the  JRC,  and  the  air 
and  surface  desk  duty  officers.  For  the  action  officers,  this  large  increase 
is  due  to  a high  number  of  users;  for  the  CCWT  duty  officers,  the  large 
increase  is  due  to  the  lengthy  sessions  each  officer  has. 


42 


MEAN  NUMBER  INSTRUCTIONS  EXECUTED  DAILY 


USER  TYPE 

Figure  6.  Mean  Number  of  Commands  (Daily)  by  User  Type 
(This  sums  the  commands  for  all  users  acting  in  the  role) 
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Of  somewhat  more  interest  are  the  increases  shown  in  numbers  of  commands 
(Fig.  6).  Modest  increases  are  shown  by  the  division/branch  administrative, 
division/branch  clerical,  J301,  and  action  officers  in  category  1.  (There  is 
no  change  for  DDO,  who  does  not  have  his  own  terminal.)  Moderate  increases 
are  shown  by  action  officers  in  category  3 and  by  air  desk  duty  officers. 

Large  increases  are  shown  by  action  officers  in  category  2 (where  many 
officers  share  a few  terminals),  surface  desk  duty  officers,  and  JRC. 

It  is  too  early  to  draw  any  useful  conclusions  from  these  comparisons  of 
loading  by  type.  However,  the  data  do  suggest  that  in  order  to  determine  load 
by  type,  these  kinds  of  comparisons  should  be  made  in  the  future,  after  the 
users  have  increased  experience  with  the  system. 

OBSERVATION 

The  pattern  of  hourly  system  use  which  has  developed  is  consistent  with 
the  normal  workday.  There  appear  to  be  different  styles  of  use  by  different 
types  of  users,  influenced  in  part  by  terminal  availability.  It  is  too  early 
to  draw  any  conclusions  about  the  level  of  use  which  can  be  expected  from 
these  different  user  types. 

DEGREE  OF  INTEGRATION  INTO  NORMAL  OPERATIONS 

The  J301  staff  is  using  the  automated  system  as  its  secondary  source  for 
message  routing.  Increasing  usage  of  the  system  by  J301  has  been  noted. 
Initially,  J301  would  log-on  in  the  morning,  route  the  message  traffic,  and 
log-off  for  the  day.  The  staff  is  using  the  system  increasingly  during  the 
day  to  route  traffic. 

The  Command  Center  and  Joint  Reconnaissance  Center  Watch  Teams  have  been 
observed  using  the  automated  system.  Some  teams  log-on  to  review  their 
traffic  when  they  come  on  watch;  then,  some  stay  logged-on  and  review  their 
traffic  on  a periodic  basis  while  others  log  on  and  off  as  they  review 
traffic.  They  create  files  on  relevant  areas  of  concern  and  file  messages 
- into  the  files.  They  have  experimented  with  the  creation  of  readboards,  but 

have  problems  because  the  data  they  require  is  not  all  on  the  MME  system. 

They  also  have  prepared  sections  of  the  morning  brief  on-line  as  text  objects. 

Many  action  officers  observed  use  the  automated  system  to  retrieve 
referenced  messages;  to  review  incoming  traffic;  to  create  text  objects  which 
later  become  sections  of  reports,  memoranda,  and  messages;  and  to  maintain 
extensive  files  relevant  to  their  jobs.  Some  have  developed  rather  precise 
selectors  to  reduce  the  date  file  to  the  few  messages  relevant  to  their  jobs, 
thus  allowing  them  to  review  the  traffic  rapidly  and  efficiently  each  day. 

The  clerical  personnel  have  been  using  the  automated  system  mainly  as  a 
source  for  message  retrieval  since  the  traffic  is  now  routed  by  the  system  to 
the  staff  officers  who  generally  file  their  own  traffic  as  they  review  it. 

The  clerical  personnel  have  also  been  observed  utilizing  the  informal  note 
capability. 
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CHANGES  IN  STAFF  OPERATIONS 


Little  change  has  been  observed  in  staff  operations  during  the  initial 
phase  of  the  experiment.  The  major  contributions  to  this  lack  of  change  are 
felt  to  be: 

(a)  The  automated  system  does  not  provide  all  of  the  functions  required 
to  perform  a user's  required  job. 

(b)  There  are  insufficient  user  terminals  on-line  to  allow  the  system  to 
interface  each  user  to  every  other  possible  user.  This  not  only 
limits  the  number  of  users  on-line,  but  requires  those  users  on-line 
to  resort  to  the  manual  system  to  interface  with  the  users  and 
organizations  not  yet  on-line. 

(c)  During  the  period  covered  by  this  report,  the  coordination  and 
release  capabilities  for  outgoing  messages  were  untested. 

TYPE  OF  USE 

Data  for  this  section  were  selected  from  data  on  SIGMA  use  at  CINCPAC 
during  the  months  of  August  and  September  1978.  This  section  describes  the 
type  of  use  made  of  the  system  by  users  in  different  roles.  Activity  is 
summarized  in  graphs  representing  data  for  four  time  periods:  19  July-2 
August,  3 August-16  August,  17  August-30  August,  and  6 September-27 
September.  SIGMA  version  2.1  was  installed  at  CINCPAC  in  early  September; 
prior  to  that  date,  version  2.0  was  in  use. 

The  graphs  (see  Appendix  A)  on  user  activity  show  the  various  types  of 
instructions  SIGMA  users  executed  while  on-line.  Because  of  the  number  of 
SIGMA  instructions,  it  was  necessary  to  group  instructions  into  the  32 
functional  categories  that  appear  on  the  two  vertical  axes  of  each  graph. 
Similar  instructions  are  grouped  together.  For  example,  the  "display  message" 
category  includes  all  instructions  and  function  keys  which  cause  a message  to 
be  displayed  such  as  "display  entry  (number)",  "redisplay  (or  show)  open 
message",  "display  message  (message-id)",  etc.  Some  of  the  categories  combine 
two  or  more  distinct  instructions  which  perform  similar  functions,  such  as 
"restrict/augment",  "file/move",  and  "copy/put/pickup/move  text". 

Each  graph  represents  the  activity  of  one  SIGMA  user  in  a specified  role 
over  an  entire  time  period  (four  graphs  for  each  role).  The  graphs  show  for 
each  instruction  category  the  percent  of  the  total  number  of  instructions 
executed  from  that  category.  They  also  show  the  percent  of  the  total 
processing  time  associated  with  the  instructions  for  that  category. 

These  percentages  are  represented  by  the  bars  on  the  graphs.  The  striped 
bars  represent  the  number  of  instructions  and  the  solid  bars  represent  machine 
processing  time.  For  example,  say  that  for  a given  user,  30Z  of  his  total 
instructions  were  from  the  "display  message"  category  and  the  processing  time 
associated  with  these  instructions  was  20Z  of  the  total  processing  time  for 
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all  instructions.  In  this  case,  the  striped  bar  at  the  "display  message"  mark 
would  extend  to  30%,  the  solid  bar  to  20Z.  In  addition,  after  each  striped 
bar  the  total  number  of  instructions  executed  from  that  category  is  included. 

On  each  graph,  the  instruction  categories  are  divided  into  two  distinct 
groups:  the  categories  in  the  left  hand  group  represent  activities  which  are 
usually  associated  with  reading  and  manipulating  incoming  messages.  The 
categories  in  the  right  hand  group  represent  activities  usually  associated 
with  outgoing  messages,  such  as  creation,  coordination,  etc.  The  right  hand 
group  also  includes  activities  associated  with  the  manipulation  of  text 
objects  and  miscellaneous  categories  such  as  "clear  view  window",  "finish", 
etc. 

A representative  sample  of  users  was  selected  from  each  of  the  roles: 
Surface,  Air,  JRC,  J301  and  from  several  of  the  other  J3  divisions  and 
branches  for  this  report.  The  division  and  branch  action  officers  were 
divided  into  three  categories  based  on  their  incoming  message  load.  Category 
1 users  are  users  with  a moderate  load  of  action  messages  and  a heavy  load  of 
information  (info)  messages.  For  Category  2 users,  the  action  message  load  is 
light  but  the  info  message  load  is  heavy.  The  action  and  info  message  loads 
are  both  light  for  users  in  Category  3 . These  categories  were  created  to 
determine  what  effect  a user's  incoming  message  load  has  on  his  use  of  SIGMA, 
for  example,  to  determine  if  users  with  heavier  incoming  message  loads  use  the 
system  more  than  users  with  lighter  incoming  message  loads.  Two  users  from 
each  category  are  included  in  our  sample  so  that  comparisons  among  the  three 
categories  can  be  made. 

Comparisons  are  also  made  between  action  officers  and  clerks.  Because  the 
total  data  set  included  very  few  clerks,  they  are  grouped  together  regardless 
of  division  or  branch.  Differences  in  system  usage  between  these  two  types  of 
users  are  noted. 

J301 

J301  is  responsible  for  the  distribution  of  all  incoming  messages  for  J3. 
Normally,  J301  logs  on  several  times  a day  and  distributes  the  messages. 
Predictably,  "route  message",  an  instruction  which  distributes  messages  to 
users  for  action  and  information,  is  the  instruction  J301  uses  most  often 
(about  25Z  of  the  commands). 

However,  there  are  several  other  SIGMA  functions  which  are  an  integral 
part  of  his  message  distribution  task.  First,  there  are  text  objects.  The 
"route  message"  instruction  takes  a special  text  object  as  one  of  its 
arguments.  This  text  object  contains  four  kinds  of  information  regarding  a 
message  to  be  distributed.  It  identifies  who  the  message  should  be  sent  to 
for  action,  who  it  should  be  sent  to  for  info,  whether  it  should  be  filed  in 
one  of  J301's  files  (other  than  his  Pending  file),  and  whether  or  not  it 
should  be  deleted  from  his  Pending  file.  (Ordinarily,  these  criteria  have 
been  determined  prior  to  a distribution  session  and  stored  in  these  routing 
text  objects.)  Some  of  J301's  time  is  spent  creating  and  editing  these  text 
objects.  All  four  actions  are  executed  by  SIGMA  whenever  a "route" 
instruction  is  executed. 
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Second,  there  is  the  use  of  selectors.  Often,  several  sets  of  messages 
will  arrive  which  have  the  same  routing  criteria.  To  route  these  messages, 
J301  uses  the  "restrict"  and  "augment"  instructions  (usually  with  stored 
selectors)  to  create  the  subsets  of  messages  with  common  routing  criteria.  He 
then  routes  all  the  messages  in  the  subset  using  a single  "route"  instruction, 
referencing  the  appropriate  text  object.  Then,  using  the  "backup" 
instructions,  he  returns  to  the  full  Pending  file  and  repeats  the  same 
procedure  on  another  set  of  messages. 

J301  cannot  always  determine  how  a message  should  be  routed  by  looking  at 
its  citation  in  the  Pending  file;  he  needs  to  look  at  the  text  of  the 
message.  When  this  is  the  case,  he  puts  the  message  on  the  screen,  using 
"display  message"  or  "view  message".  Usually  he  can  determine  how  the  message 
should  be  routed  after  reading  the  first  few  lines  of  the  text  field.  Most  of 
the  time,  though,  the  routing  criteria  can  be  determined  from  the  message's 
Pending  file  citation,  as  the  graphs  demonstrate;  "display  message"  and  "view 
message"  instructions  constitute  only  a small  percentage  (2-3Z)  of  all  of 
J301's  activity. 

Action  Officers 

Action  officers  are  grouped  into  categories  based  on  their  division  or 
branch  incoming  message  load  (see  introduction  to  this  section).  For  Category 
1,  data  from  J311  and  J313  were  selected.  For  Category  2,  J34  and  J342  were 
selected,  and  for  Category  3,  J315  and  J32  were  selected.  Category  1 users 
have  the  heaviest  incoming  message  loads,  followed  by  Category  2 and  Category 
3,  which  has  the  lightest  load. 

Because  the  time  period  selected  for  this  report  was  the  intial  period  of 
limited  experimental  use  of  the  system,  users  of  SIGMA  operated  at  different 
levels  of  proficiency.  In  other  words,  some  users  were  very  familiar  with  the 
system,  executing  a wide  variety  of  instructions,  while  other  users  executed 
only  a few  basic  instructions,  such  as  "display  message"  and  "display  file". 
This  fact  was  particularly  true  of  action  officers,  making  it  difficult  to 
make  comparisons  between  the  three  categories.  Often,  within  a category, 
users  from  one  role  appeared  to  be  active,  experienced  users,  while  users  from 
another  role  executed  a small  variety  of  instructions.  Because  of  this,  it  is 
impossible  to  generalize  the  characteristics  of  SIGMA  users  in  a single 
category.  As  the  system  becomes  familiar  to  all  the  users,  we  might  expect 
users  in  a given  category  to  have  similar  patterns  of  use  which  might  differ 
to  some  extent  from  patterns  of  use  observed  in  users  from  other  categories. 

Action  officers,  as  a group,  have  some  similarities  in  the  way  they  use 
SIGMA,  despite  the  discrepancies  in  their  degrees  of  proficiency.  All  of  them 
read  their  incoming  messages  by  displaying  their  Pending  files  (15-20Z  of 
their  total  commands)  and  then  reading  the  messages  of  interest.  Some  action 
officers  prefer  to  use  the  display  window  for  reading  their  messages  while 
others  prefer  the  view  window.  These  preferences  are  consistent;  users  used 
one  window  or  the  other  almost  exclusively.  In  this  mode  of  operation,  the 
advantage  of  using  the  view  window  is  that  the  Pending  file  can  be  displayed 
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simultaneously.  The  advantage  of  the  display  window  is  that  the  message 
replaces  the  file  as  the  only  object  on  the  screen  (assuming  that  there  is 
nothing  in  the  view  window  at  the  same  time),  and  it  is  displayed  in  brighter 
video.  Other  frequently  used  instructions,  such  as  "restrict",  "augment", 
"backup"  and  "find  entry",  are  aids  to  a user  trying  to  read  his  incoming 
messages  (15-20Z). 

Host  action  officers  with  at  least  a moderate  level  of  activity  do  some 
file  maintenance  such  as  filing,  moving,  and  deleting  messages.  Instructions 
in  the  "create  file",  "comment  message”  and  "keyword  message"  categories  might 
also  qualify  as  file  maintenance.  In  addition,  a small  amount  of  message 
creation  activity  was  observed  in  the  data  from  these  action  officers.  Most 
likely,  the  messages  were  informal,  internal  notes  to  other  CINCPAC  users,  as, 
at  the  time  these  data  were  collected,  formal  coordination  and  release  of 
outgoing  messages  had  not  yet  been  implemented. 

Clerks 

There  was  not  much  data  from  clerks  during  the  time  period  these  data  were 
collected,  and  not  each  category  was  represented  in  the  group  of  clerks,  so 
all  the  clerks  were  grouped  together,  regardless  of  category.  There  were  many 
similarities  among  the  activities  of  all  the  clerks,  so  we  are  justified  in 
grouping  them  together.  In  fact,  in  several  cases,  one  clerk  was  logged  on  as 
more  than  one  role,  so  that  in  grouping  clerks  over  roles,  we  were  often 
grouping  data  from  the  same  clerk. 

The  clerks'  activities  did  not  differ  significantly  from  those  of  the 
action  officers,  although  they  did  seem  to  execute  fewer  commands  in  general. 
Their  primary  activity  was  displaying  or  viewing  messages  in  their  message- 
files,  followed  by  file  maintenance  ("file/move"  and  "delete  message" 
instructions).  As  with  the  action  officers,  the  clerks  used  "restrict/ 
augment"  and  "backup"  instructions  as  an  aid  to  reading  their  messages.  There 
was  a small  amount  of  message  creation  done  as  well. 

In  the  future,  clerks  may  be  expected  to  use  SIGMA  differently  than  action 
officers,  and  these  differences  would  reflect  the  differences  that  exist 
between  a clerk's  duties  and  an  action  officer's  duties.  However,  the  data 
collected  for  the  two  groups  during  this  period  of  limited  experimental  use  of 
SIGMA  do  not  reflect  those  differences. 

Surface  and  Air 

The  Surface  and  Air  roles  are  users  who  are  duty  officers  in  the  Command 
Center  at  CINCPAC.  These  two  roles  receive  copies  of  almost  all  the  incoming 
messages  for  CINCPAC.  On  SIGMA,  these  users  receive  these  messges  in  their 
Pending  files.  They  scan  the  message  citations  in  this  file  daily  looking  for 
reports  of  situations  and  events  requiring  immediate  action  and/or  the 
attention  of  designated  action  officers.  Messages  of  importance  are  displayed 
or  viewed  and  sometimes  forwarded  to  the  appropriate  person.  Most  other 
messages  are  deleted,  one  at  a time  as  the  user  scans  his  file.  This  fact  is 
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reflected  in  the  graphs.  There  are  a fairly  substantial  number  of  "display 
message"  or  "view  message"  instructions  (10-202)  and  an  even  greater  number  of 
'd*l*te  message"  instructions  (40-602).  This  indicates  that  a Surface  or  Air 
user  is  most  likely  scrolling  through  his  Pending  file,  displaying  (or 
viewing)  messages,  then  filing  and/or  deleting  them,  based  on  the  information 
contained  in  the  message's  citation.  Since  "delete  message"  is  overwhelmingly 
the  most-used  instruction,  it  is  likely  that  Surface  and  Air  get  a great  many 
messages  that  are  of  little  interest  to  the  CCWT.  A large  variety  of  other 
instructions  are  executed  by  Surface  and  Air  users,  but  those  instructions 
make  up  only  a small  part  of  their  overall  activity.  These  activities  include 
message  and  file  creation,  printing,  and  message  forwarding. 

JRC 

Users  in  the  JRC  role  work  in  the  Joint  Reconnaissance  Center  at  CINCPAC; 
JRC  is  responsible  for  seeing  that  reconnaissance  missions  are  carried  out 
correctly.  Their  use  of  SIGMA  is  much  like  that  of  Surface  and  Air  users. 

They  receive  most  of  the  incoming  messages  for  J3,  but  are  primarily  concerned 
with  those  messages  which  relate  to  their  jobs.  Therefore,  "delete  message" 
is  a commonly  used  instruction  (20— 502)  followed  by  "view  message"  and 
display  message  (15-252).  Presumably,  the  mode  of  operation  is  to  display 
the  Pending  file  ("display  file"  instructions  make  up  about  15-202  of  the 
total)  and  then  scroll  through  it,  reading  messages  of  interest  to  JRC  and 
deleting  messages  which  are  not  of  interest.  "Res trie t/augment"  instructions 
(about  52)  are  used  to  aid  JRC  in  his  file  scanning. 

OBSERVATIONS 

From  these  observations,  it  is  obvious  that  a user's  job  does  influence 
the  way  he  uses  SIGMA,  what  instructions  he  executes,  what  SIGMA  functions  he 
takes  advantage  of.  J301,  whose  primary  job  is  message  distribution,  uses  the 
system  quite  differently  than  a user  from,  say,  the  Command  Center  Watch  Team. 

There  are  many  differences  between  individuals  in  the  same  role  which  can 
be  attributed  to  two  facts.  One  is  that  different  users  have  different 
degrees  of  proficiency  with  the  system.  The  second  reason  is  that  the  paper 
system  is  still  the  primary  message  handling  system  in  use  at  CINCPAC.  While 
users  are  encouraged  to  make  use  of  the  computer-aided  system  for  the  daily 
message  traffic,  many  may  still  rely  on  the  paper  system.  Once  the  official 
experiment  begins,  it  is  expected  that  these  individual  differences  may  be 
minimized  and  that  users  with  similar  work  loads  will  be  using  SIGMA  in 
s imi 1 ar  manne  r s . 
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It  is  important  to  reemphasize  that  the  data  collection  to  support  this 
analysis  was  done  very  early  in  the  experiment;  therefore,  the  users' 
impressions  must  be  measured  very  carefully  against  their  experience,  the 
system  capability,  the  system  reliability,  and  the  system  load.  For  instance, 
the  time  covered  by  the  questionnaire  was  one  in  which  there  was  not  a heavy 
load  on  the  system.  Thus,  a user's  impression  concerning  the  adequacy  of  the 
response  time  may  change  when  the  system  use  increases.  Also,  the  system  is 
still  undergoing  changes;  thus,  a user's  impression  that  the  message  archive 
facility  is  inadequate  is  valid  for  the  time  of  the  survey,  but  may  not  be 
valid  after  a new  archiving  procedure  is  implemented.  There  exists  a problem 
in  either  the  hardware  or  software  that  causes  a user's  terminal  to  be 
disconnected  abruptly  from  the  message  service  system.  This  can  result  in 
loss  of  the  work  that  has  been  accomplished  during  the  session.  This  is  a 
major  problem  for  the  users,  and  a solution  is  being  pursued  vigorously  by  the 
system  developers.  When  the  solution  is  found,  the  users'  perceptions  of  the 
system  reliability  probably  will  change. 

The  subjective  evaluation  of  the  MME  is  based  on: 

(a)  the  replies  to  the  survey  questionnaire  (polled  J3  users  the  week  of 
13  November  1978  concerning  their  use  of  the  system); 

(b)  observations,  conversations,  and  interviews  with  the  users;  and 

(c)  the  data  collected  that  are  described  in  Section  5. 

A sample  of  the  survey  questionnaire  is  included  in  this  section.  The 
application  of  the  questionnaire  and  the  tabulation  of  the  results  was  a joint 
effort  by  the  on-site  team  including  personnel  from  the  J6  and  the  J3 
directorates.  The  purpose  of  the  questionnaire  was  to  characterize  each 
user's  use  of  the  MME  system  (frequency  and  functions),  to  determine  each 
user's  perception  of  the  system,  and  to  identify  those  functions  that  each 
user  sees  as  constraining.  J341  categorized  the  users'  responses  to  the 
survey  into  three  groups.  The  details  of  the  questionnaire  and  a discussion 
of  the  results  are  contained  in  the  section  of  this  report  prepared  by  the 
CINCPAC  personnel. 

The  following  observations  summarize  the  information  derived  from  all  the 
sources  cited  in  this  report. 

(a)  The  hardware  and  software  have  been  undergoing  significant  changes. 

Neither  the  users  nor  the  evaluators  as  of  yqt  have  a baseline  system 
to  judge;  neither  has  J3  built  up  sufficient  confidence  in  the  system 
to  rely  on  it  for  full  message-handling  support. 
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(b)  A great  deal  of  the  information  (e.g.,  operational  plans,  memos, 
reports)  needed  by  a user  in  preparing  a message  is  not  available  on 
the  MME  system.  This  limitation  must  be  considered  when  extrapolat- 
ing the  results  of  the  MME  to  other  staff  environments. 

(c)  The  system  has  been  in  only  a limited  experimental  use  (LEU)  phase. 
During  the  period,  there  have  been  occasional  system  failures  that 
caused  users  to  lose  data.  Further,  there  have  not  been  enough 
terminals  available.  As  a result,  many  users  have  not  converted  all 
of  their  message-processing  tasks  to  the  MME  system. 

(d)  Step-by-step  user  guides  for  various  functions  (releasing  messages, 
reading  incoming  messages,  composing  and  sending  notes)  appear  to  be 
effective  in  encouraging  system  use. 

(e)  The  features  most  used  to  date  have  been  message  routing,  filing,  and 
retrieval.  The  routing  of  messages  to  those  users  who  have  terminals 
has  become  an  almost  automated  routine.  J301  initiates  a series  of 
machine  instructions  to  route  the  traffic,  waits  for  the  MME  system 
to  complete  the  work,  and  then,  still  using  the  MME  system,  routes 
any  remaining  messages.  Using  this  procedure,  the  time  for  routing 
messages  can  be  decreased  from  approximately  four  hours  with  the 
manual  system  to  approximately  one  hour  using  the  MME  system.  The 
major  problem  is  that  all  the  users  served  by  J301's  routing  do  not 
have  terminals;  hence,  the  manual  system  cannot  be  replaced 
completely.  When  filing,  information  copies  of  messages  can  be 
retrieved  by  users  directly  from  the  file  by  means  of  various  message 
selectors.  Thus,  the  information  copies  of  messages  need  not  be 
routed  explicitly  by  J301;  rather,  he  can  rely  on  the  users  using 
their  own  specialized  retrieval  criteria  for  the  messages.  In 
addition,  the  users  utilize  the  file-building  and  message-retrieval 
capabilities  for  their  personal  and  organizational  files. 

(f)  The  comnand  center  and  joint  reconnaissance  center  (JRC)  watch  teams 
are  using  the  MME  system  to  review  their  traffic  and  to  create  files 
that  reflect  particular  areas  of  concern.  They  then  file  relevant 
messages  in  them  for  use  in  creating  readboards  and  preparing  sections 
of  the  morning  brief. 

(g)  Some  action  officers  are  using  the  MME  system  to  review  incoming 
traffic;  to  retrieve  referenced  messages;  to  create  text  objects  as 
bases  for  sections  of  reports,  memoranda,  and  messages;  and  to 
maintain  large  files  relevant  to  their  duties. 

(h)  The  clerical  personnel  are  using  the  MME  system  mainly  for  message 
retrieval  because  the  traffic  is  routed  directly  to  the  staff 
officers  for  their  disposal;  they  also  utilize  the  informal  note 
capability. 

(i)  The  strict  enforcement  of  a security  policy  using  the  concept  of  a 
security  kernel  does  not  appear  to  have  added  undue  restrictions  on 
the  user  interface. 
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Training  Lessons  (summarized  from  Section  6) 

(j)  Although  the  on-line  lessons  have  proved  to  be  very  useful,  they  have 
not  been  accepted  to  the  degree  that  we  had  hoped. 

(k)  Much  personal  attention  is  needed  by  the  trainees. 

(l)  The  primers  are  used  more  than  the  reference  manuals.  There  are 
probably  two  reasons  for  this  - the  primer  is  written  in  a more 
conversational,  breezy  style,  and  at  the  present  stage  of  the  experiment, 
introductory  level  material  is  needed  more  than  the  details  of  features 
and  instructions. 

(m)  The  lack  of  system  stability  has  had  a deleterious  effect  on  the 
training  of  the  users. 

(n)  Having  users  participate  in  system  testing,  such  as  the  load  tests, 
has  had  a beneficial  effect  on  training. 

* 

(o)  User  training  suffered  from  a number  of  "false  starts"  for  the 
experiment.  Some  users  commenced  their  lesson-taking  as  much  as  five  or 
six  months  in  advance.  Not  all  users  suffered  from  this  delay,  but 
certainly  for  some,  the  momentum  developed  during  lesson-taking  was 
dissipated  before  the  limited  use  phase  began. 

(p)  One  aspect  of  the  MME  environment  that  has  caused  more  problems  than 
anticipated  is  that  of  lack  of  free  access  to  some  of  the  work  spaces  in 
which  the  terminals  are  installed.  This  comes  about  because  of  the  high 
security  level  at  which  some  of  the  offices  operate.  Although  some 
project  personnel  have  the  necessary  clearances,  others  do  not,  and  on 
occasion  someone  not  completely  familiar  with  a problem  may  be  the  one  who 
has  to  respond. 

Inferences  from  User  Questionnaires  (discussed  in  detail  on  pages  56 
through  59). 

(q)  Those  who  most  frequently  use  the  message  distribution  function  have 
a generally  good  perception  of  it. 

(r)  Most  users  generally  achieve  their  desired  results  and  perceive  very 
little  limitation  in  the  use  of  the  message  reading  and  filing 
capability.  Members  of  the  command  center  watch  team  (CCWT)  report 
problems  with  this  function  and  have  a noticeably  less-favorable 
perception  and  foresee  limitations  (possibly  unacceptable  ones). 

(s)  The  users  generally  are  able  to  retrieve  the  messages  that  they  need, 
but  many  report  problems  in  the  retrieval  of  archived  messages. 

• (t)  The  capabilities  for  text  preparation  and  message  drafting  have  been 

used  almost  exclusively  by  the  Group  A users  who  were  fully  trained.  The 
users  rate  these  functions  highly  and  perceive  no  system  limitations  in 
t . their  use. 
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(u)  The  informal  note  function  has  had  limited  use;  it  is  liked  by  the 
users. 

(v)  The  coordination  and  release  functions  have  not  been  used. 

(w)  Some  users  see  the  slow  response  time  as  a problem  (reported  "some 
limit");  some  viewed  it  as  usually  satisfactory. 

CINCPAC's  EVALUATION  OF  THE  MME  SYSTEM 

The  remainder  of  this  section  was  written  by  the  CINCPAC  staff  to  describe 
the  user's  perception  of  the  MME  system. 

Summary 

This  section  provides  CINCPAC's  evaluation  of  the  MME  system  during  the 
period  July  1978-November  1978,  during  which  time  approximately  half  of  the 
potential  J3  users  utilized  the  system  either  to  support  their  assigned  duties 
or  for  training  and  familiarization.  At  the  conclusion  of  this  period,  they 
were  asked  to  report  their  preliminary  observations  and  perceptions  of  the 
system  through  a user  questionnaire.  Their  overall  evaluation  of  the  system 
was  that  the  MME  was  satisfactory  in  most  aspects.  Corrections  of  problems 
and  limitations  identified  by  the  users  are,  for  the  most  part,  scheduled  for 
future  system  releases.  Since  the  number  of  users  and  scope  of  their  use  were 
limited,  these  initial  observations  need  to  be  confirmed  during  Full 
Experimental  Use  (FEU)  of  the  MME. 

Observation  Environment 

User  population 

Ninety-nine  members  of  the  J3  staff  have  been  identified  as  candidates  to 
participate  in  the  experiment  as  users  of  the  system.  Forty-four  of  these  had 
sufficient  MME  training  and  experience  at  the  time  the  user  questionnaire  was 
distributed.  A detailed  outline  of  the  polled  population  is  provided  in  this 
section. 

IZPe_  of  use 

During  the  observation  period,  J3  personnel  used  the  MME  in  parallel  with 
existing  manual  message  handling  procedures.  This  phase  of  system  development 
was  entitled  Limited  Experimental  Use  (LEU)  and  directed  certain  minimal  use 
of  the  system  for  applicable  J3  users.  The  objective  of  the  LEU  was  to 
develop  J3  procedures  for  eventual  use  of  the  MME  as  the  primary  message 
handling  system,  and  to  provide  practice  in  using  several  different  system 
capabilities.  Several  users  voluntarily  intensified  their  usage  during  LEU 
and  came  to  depend  upon  the  MME  as  their  primary  message  handling  tool. 

System  status 

There  were  significant  system  upgrades  in  both  hardware  and  software 
during  the  evaluation  period.  The  users'  observations  are,  therefore, 
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oriented  to  the  system  as  it  existed  during  the  latter  part  of  this  period 
which  included  a new  KL  processor  with  over  one  million  words  of  memory  and  14 
user  terminals. 
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User  Perceptions  and  Acceptance  of  MME 

Questionnaire 

User  observations  of  the  system  were  evaluated  on  the  basis  of  a 
questionnaire.  A copy  is  included  at  the  end  of  this  section.  The 
questionnaire  was  designed  to: 

(a)  Characterize  the  frequency  with  which  each  respondent  used  the  system 
and  each  of  its  major  functions; 

(b)  Determine  whether  a respondent  perceives  each  major  MME  function  as 
being  helpful  to  him;  and 

(c)  Identify  those  functional  capabilities  which  are  thought  to  limit  the 
usefulness  of  the  system. 

User  sample 

During  the  week  of  13  November  1978,  forty-four  J3  users  of  the  MME  were 
asked  to  complete  the  questionnaire.  Responses  were  received  from  thirty-nine 
(90?)  of  the  users.  The  users  polled  were  those  who  made  most  frequent  use  of 
the  MME.  The  SO  or  more  potential  users  who  have  not  used  the  system,  whether 
from  lack  of  terminals,  lack  of  opportunity,  or  lack  of  interest,  were  not 
polled.  The  44  users  are  subdivided  into  the  following  three  groups: 

(a)  Group  A consisted  of  18  people  judged  by  J341  as  being  trained  since 
they  had  completed  all  of  the  milestones  established  by  J341.  Seventeen  of 
this  group  returned  the  questionnaire.  One  of  the  17  did  not  respond  to  most 
questions  since  his  terminal  had  been  out  of  service  since  early  August. 

(b)  Group  B consisted  of  nine  persons  considered  to  be  partially 
trained.  Of  the  eight  who  completed  the  questionnaire,  one  did  not  answer 
most  questions  as  his  use  was  very  limited. 

(c)  Group  C consisted  of  17  persons  judged  as  marginally  trained.  Of  the 
14  who  returned  the  questionnaire,  three  did  not  answer  most  questions  because 
of  insufficient  use. 

Tabular  results 

Tables  4,  5,  and  6 present  the  responses  for  each  user  broken  down  by  the 
three  groups  identified  above.  In  presenting  these  results,  numerical  values 
have  been  given  to  responses  in  sections  1 and  2 of  the  questionnaire  which 
deal  with  frequency  of  system  use.  These  numbers  are  not  scaled  valuations  of 
usage;  they  are  simply  ranking  numbers.  The  larger  values  imply  more  frequent 
use.  Alphabetic  coding  has  been  given  to  responses  in  sections  3 and  4 which 
deal  with  user  perceptions  and  acceptance.  These  codes  are: 
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Excellent 


r ? 


S ■ Satisfactory 
P ■ Poor 

U * Unsatisfactory 

Discussion  of  results 

The  following  discussion  summarizes  the  users'  responses  for  each 
functional  capability  and  highlights  some  of  the  best  and  least  liked 
features.  Quotations  from  several  users  have  been  included  to  give  the  reader 
a more  direct  impression  of  their  attitudes.  Phrases  included  in  square 
brackets  within  the  quotations  have  been  inserted  to  clarify  the  meaning  of 
the  comment  when  taken  out  of  its  context.  While  reading  this  discussion,  the 
reader  should  keep  in  mind  that  these  users  were  selected  as  those  having  most 
exposure  to  the  MME  and  that  most  of  their  use  of  the  system  was  ancillary  to 
the  manual  message  system. 

(1)  Distribution  function.  Those  who  most  frequently  use  the 
distribution  capability  (primarily  J301  and  some  action  officers)  have  a 
generally  good  perception  of  it.  A potential  limitation  foreseen  by  a key 
user  is  that  terminals  have  not  been  allocated  to  J35,  J36,  and  J37 . He 
stated,  "Difficult  to  substitute  MME  for  paper  system  when  all  Divisions  do 
not  have  terminals."  Less  frequent  users  of  this  capability  see  little 
limitation  to  its  usefulness.  In  evaluating  the  distribution  capability, 
there  is  little  difference  among  Groups  A,  B,  and  C.  However,  Command  Center 
officers  report  almost  no  use  of  this  function.  Although  action  officers  use 
it  much  less  than  do  NCOs,  their  perception  and  acceptance  are  favorable. 

(2)  Reading  and  filing  messages.  Both  frequent  and  occasional  users  of 
the  MME  achieve  desired  results  and  see  very  little  limitation  in  this 
capability  except  for  members  of  the  CCWT  (command  center  watch  team).  They 
have  a noticeably  less  favorable  perception  and  foresee  limitations,  sometimes 
unacceptable  ones. 

Respondent  #8  "...reading  all  msgs  contained  in  the  pending  file  is  a very 
(CCWT)  slow  and  laborious  process  and  has  no  advantage  over  the  time 

honored  system  of  reading  hard  copies.  ...during  crisis... the 
time  factor  becomes  unacceptable." 

"Scanning. .. is  very  fast,  but  with  disadvantages. . .it  becomes 
rather  risky  to  decide  on  the  basis  of  fthe  message  citation] 
whether  to  read  or  delete  [a  message] ." 

Respondent  #40  "Messages  in  freadboard]  files  are  filed  in  random  sequence 
(CCWT)  rather  than  in  DTG  order." 

(3)  Retrieval . The  retrieval  capabilities  have  been  used  by  all  groups. 
Users  report  that  they  can  "usually"  to  "almost  always"  get  the  message  they 


56 


i 


jj 


need,  but  many  users  report  problems  and  limitations  in  retrieving  from  the 
archive.  Their  complaints  are  usually  that  the  time  to  retrieve  such  messages 
is  much  too  long. 


Respondent  #3  (Considered  retrieval  a serious  limit)  "mostly  due  to  message 
not  in  files  due  to  classification  or  comeback  copies  not 
filed."  (Note:  The  problem  of  nonreceipt  of  comeback  copies 
will  be  corrected  by  a new  procedure  in  early  December.) 

Respondent  #5  "Multi-part  messages  not  fully  retrieved." 

Respondent  #6  "Need  TS  [messages]." 

"A  good  action  officer  feature  is  the  date  file  for  immediate 
retrieval." 

Respondent  #19  "a  wait  of  over  5-10  minutes  is  unacceptable  as  you  don't 

need  that  much  time  to  find  a message  in  the  [manual] 
filing  cabinet." 

Respondent  #8  "retrieving  from  the  datefile,  by  either  DTG , originator  or 
subject  is  a superb  feature... of  considerable  use  to  me." 
"retrieving  from  text  is  outstanding."  "the  majority  [of  my] 
files  are  created  for  long  term  usage,  which  is  incompatible 
with  the  30-day  rapid  access  file."  (because  retrieval  from 
archive  was  so  slow) 

Respondent  #40  "frequently  referenced  messages  cannot  be  located  in  the 

MME  datefile." 

Respondent  #27  "very  good  for  retrieving  messages,  don't  have  to  call 

comm,  ctr." 


(4)  Preparation  of  text  and  drafting  messages.  These  capabilities  have 
been  used  almost  exclusively  by  Group  A users;  they  were  rated  highly  by  them 
and  seen  as  no  limitation. 


Respondent  #8  "drafting. .. is  another  strong  feature. . .associated  capability 
for  internal  routing  and  chopping  is  outstanding." 

"editing. .. is  rather  simple  and  far  superior  to  the  standard 
typewriter." 

"a  weakness ... is  printing!  ...MME  system  often  refuses  to 
retain  or  accept  the  format." 

Respondent  #40  "ability  to  make  rapid  corrections ... is  a major  advantage." 

Respondent  #15  "formatting  routine  gives  problems  thru  reformatting  of  the 

text . " 
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Respondent  #18 


Respondent  #30 


"...biggest  irritant  to  me  is  the  reformatting"  (but  he  did 
not  view  preparation  as  a limitation). 

(sees  preparation  as  an  unacceptable  limitation)  "for 
non-clerk  typist." 


(5)  Sending  notes.  Based  on  limited  usage,  this  capability  is  well 
perceived  and  seen  as  no  limitation. 

(6)  Coordination  and  release.  During  the  period  covered  by  this  report, 
J34  asked  J3  users  to  do  some  message  coordination  up  to  the  point  of  release, 
but  not  to  release  messages  on  the  MME.  The  capability  was  given  insufficient 
usage  to  warrant  but  few  specific  comments. 


Respondent  #3 


Respondent  #13 
Respondent  #35 


Respondent  #40 


"message  distribution  and  coordination  will  function  better 
when  more  people  get  terminals" 

"face-to-face  is  still  best" 

(sees  a serious  limit  in  coordination  and  release) 
"non-sequential  chop;  release  before  chop  possibility;  lack 
of  reference  material  with  message  [printed  pubs,  etc.]." 

"messages  sent  for  chop  return  to  the  originator  without 
the  chopper's  changes  having  been  incorporated.  ...very 
easy  to  overlook  wordsmithing  and  small  changes." 


(7)  Response  time.  Experienced  users  checked  that  response  time  to 
instruction  is  "usually  satisfactory"  (50%  to  80%  of  the  time)  and  viewed  it 
to  have  "some  limit"  during  the  experiment.  The  inexperienced  users  report 
more  problems  and  see  more  limitation.  Perhaps  experienced  users  have  spent 
more  time  on  the  system  and  have  felt  the  improvement  resulting  from  the  KL 
installation.  Comments  concerned  with  the  response  time  for  archive 
retrievals  have  been  mentioned  under  Retrieval.  The  following  comments  are 
directed  only  toward  response  time  to  instructions: 


Respondent  #8 


Respondent  #9 


Respondent  #16 
Respondent  #42 


"response  time  to  instructions  is  generally  adequate. 

I... can  discern  no  appreciable  improvement"  (from  the  KL 
system)  "seems  to  be  little  correlation  between  ... 
responsiveness  and  thi  displayed  load  average.  The  system 
can  be  sluggish  with  a load  average  of  2.0,  and  conversely 
operate  normally  with  an  8.0  load  average." 

"Time  delay  seems  to  have  increased  since  new  installation 
even  with  low  LDAV." 

"Response  time  too  long" 

"Response  time  to  instructions  is  generally  satisfactory. 
During  periods  of  high  volume  transfer. . .the  MME  has 
serious  and  unacceptable  limitation." 
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(8)  Reliability.  Most  respondents  encountered  problems  in  this  area  and 
foresaw  some  limit  on  system  usefulness  as  a result. 


Respondent  #8 


Respondent  #40 
(CCWT) 


Respondent  #16 
Respondent  #27 


Respondent  #31 


Respondent  #34 


"overall  reliability  is  good"  "amount  of  downtime  is 
rather  excessive.  Taking  the  system  down  for  periods  of 
eight  hours  or  so  (for  preventative  maintenance)  is  not 
realistic  and  totally  unacceptable  to  users.  Downtime  for 
routine  maintenance  has  to  be  reduced  significantly." 

"printer. . .marginally  satisfactory" 

"before  the  new  processor. . .system  hung  and  crashed 
frequently 

...will  still  do  you  in  if  you  don't  take  time  to  do  a 
FINISH  every  once  in  awhile  to  protect  work." 

"the  current. . .maintenance  schedule  requiring  8 hours  of 
downtime  every  Wednesday  afternoon  (1400-2200)  is  not 
acceptable. . .a  few  hours  during  the  day  from  say  1000-1400, 
two  or  three  times  a week  would  be  better.  If  a long  time 
is  needed,  Saturday  PM  or  Sunday  morning... if  fthe  MME]  is 
to  become  a way  of  life  for  the  CCWT." 

(Called  reliability  unacceptable)  "system  crashes  too  often" 

"poor  reliability  during  a critical  time  would  render  the 
system  useless..." 

"until  users  realize  that  the  MME  is  not  going  to  go  down 
repeatedly  there  will  continue  to  be  a resistance  to  using 
it" 

"1st  priority  is  to  improve  reliability  of  MME;  we  have 
frequent  hangs  and  crashes." 


Summary 


(1)  Overall,  the  system  is  perceived  as  being  useful.  There  are  some 
limitations,  particularly  for  the  functions  of  archive  retrieval,  message 
coordination  and  reliability. 


(2)  The  system  is  best  received  by  those  who  use  it  most. 


(3)  The  CCWT  officers,  in  consideration  of  their  need  to  handle  large 
numbers  of  messages  in  crisis  and  exercises,  are  least  accepting  of  the 
various  groups  who  were  polled.  As  in  other  groups,  the  most  experienced  were 
more  favorably  inclined. 
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Additional  CINCPAC  Comments 

Most  substantive  user  observations  and  perceptions  are  presented  in  the 
previous  paragraphs.  This  section  includes  additional  management  observations 
to  focus  the  attention  of  the  MME  team  on  certain  capabilities  which  may  limit 
the  usefulness  of  the  system. 

a.  Suspected  shortfalls 

(1)  Access  to  the  system.  At  the  beginning  of  the  experiment,  a 
decision  was  made,  with  the  agreement  of  J3,  to  concentrate  the  limited  number 
of  terminals  available  within  certain  key  divisions.  Consequently,  divisions 
J35  (exercises),  J36  (Special  Operations),  and  J37  (geophysics;  i.e.,  Weather) 
do  not  have  access  to  MME  terminals  and  are  not  users.  These  divisions 
receive  about  25Z  of  the  J3  action  messages  (source:  initial  draft  of 
Baseline  Data  Repaort).  Their  lack  of  participation  may  become  a limitation 
on  assessing  the  overall  acceptability  of  the  system. 

(2)  Limited  numbers  of  terminals.  A critical  period  for  message 
review  is  in  the  early  morning  before  staff  meetings.  Only  24  terminals  are 
currently  planned  to  serve  about  100  users  and  some  of  these  terminals  are 
reserved  for  one  or  two  users  (e.g.,  J3,  J30).  There  may  be  an  insufficient 
number  of  terminals  to  permit  all  the  users  to  rely  effectively  on  the  MME  as 
their  primary  message  handling  tool  during  these  active  morning  hours. 

b.  Suggested  areas  for  further  analysis 

(1)  Review  of  message  files.  Senior  officers  and  CCWT  officers  in 
the  Command  Center  Watch  Teams  currently  thumb  through  large  numbers  of 
messages  in  a few  minutes  and  quickly  select  those  which  appear  to  be 
important.  Some  feel  that  attempting  to  judge  importance  of  a message  on  the 
basis  of  the  brief  MME  citation  is  risky.  A more  careful  examination  of  a 
message  requires  a DISPLAY  command  and  (possibly  extensive)  scrolling  to  the 
text.  These  tasks  take  much  more  time  than  thumbing  through  paper  copies. 

This  slowness,  reported  as  a problem  by  CCWT  officers,  may  become  critical 
both  for  the  CCWT  and  for  division  chiefs  during  Full  Experimental  Use. 

(2)  Preparation  of  outgoing  messages.  The  drafting  and  editing  of 
messages  on  the  MME  is  reported  as  satisfactory  by  most  action  officers.  One 
senior  officer,  however,  identified  a problem  likely  to  be  faced  by  senior 
officers,  few  of  whom  are  good  typists.  Making  editorial  changes,  suggestions 
and  comments  (particularly  since  comments  cannot  be  put  exactly  where  desired) 
is  much  slower  and  involved  on  a terminal  than  writing  on  a hard  copy.  For 
this  task,  it  is  likely  that  such  officers  will  often  use  MME  printer  output 
rather  than  the  terminal  as  the  more  desirable  way  to  edit  draft  messages. 
Since  several  users  have  reported  problems  with  the  printers  (odd-feeling 
paper,  poor  quality  print,  strange  reformatting),  these  characteristics  may 
affect  acceptability  of  the  system  by  senior  officers. 

(3)  Coordination.  The  coordination  function  was  little  used  during 
the  report  period.  However,  several  users  did  identify  serious  problems,  not 
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all  of  which  will  be  corrected  in  the  January  1979  SIGMA  release.  Since  it  is 
impossible  to  judge  the  effectiveness  of  the  revised  coordination  features  at 
this  time,  this  area  needs  to  be  monitored  carefully. 


c.  Conclusion 


Some  of  the  problems  identified  by  users  and  analyzed  in  the  last  few 
paragraphs  are  already  being  corrected.  An  early  evaluation  of  the 
effectiveness  of  the  corrections  is  needed  so  that  prompt  changes  can  be  made 
if  the  early  evaluations  reveal  difficulties  more  serious  during  Full 
Experimental  Use. 


i 


USER  QUESTIONNAIRE 

The  following  is  a reproduction  of  the  questionnaire  that  was  distributed 
to  the  users. 


MME  USER  QUESTIONNAIRE 


Introduction 

The  MME  Program  Managers  are  writing  a preliminary  evaluation  of  the  MME  - 
a Quick  Look  Report  to  help  guide  the  future  development  of  the  experiment  and 
to  serve  as  a progress  report  to  the  sponsors  and  to  the  staff  of  the 
Assistant  Secretary  of  Defense  for  Command,  Control,  Communications  and 
Intelligence.  This  is  one  of  only  3 such  reports. 

An  essential  section  of  this  report  concerns  the  reactions  of  the  users. 
The  questionnaire  below  is  intended  to  obtain  those  reactions  in  a form  that 
can  be  readily  summarized  for  top-level  management.  We  are  trying  to 
determine  how  each  of  you  is  using  the  MME,  how  you  perceive  its  major 
capabilities  as  helping  (or  hindering)  your  job,  and  how  well  you  will  be  able 
to  use  it  during  the  next  year. 

Consequently,  your  thoughtful  responses  to  the  questions  below  are  of 
considerable  importance  to  us  and  to  the  MME  program.  Please  use  the  Remarks 
column  to  include  specific  gripes,  requests,  and  plaudits  (if  any). 
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.... 


1 


Name 


USE  OF  THE  MME 


Have  you  been  using  the  MME  since  July: 

a.  to  support  your  regular  job? 


occasionally 


frequently  (over  3 times /wk) 


b.  to  support  the  experiment? 


occasionally 


frequently  (over  3 times/vk) 


If  you  answered  NO  to  both  questions,  please  indicate  why. 


too  busy 


not  trained 


terminal  not  handy 


other 


and  do  not  complete  the  rest  of  the  questionnaire. 


HOW  DO  YOU  USE  THE  MME? 


sometimes  often 

(less  than  (3/wk  or 
3/wk)  more) 


Remarks 


- to  forward /distribute  msgs 

- to  scan/read/file  msgs 

- to  retrieve  msgs  from 

your  own  files 
from  the  datefiles 

- to  prepare  reports  & 

working  papers 

- to  draft  msgs /memos 


- to  send  notes  to  users 
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f 


YOOR  PERCEPTIONS  OF  THE  SYSTEM 


(if  you  do  not  use 

occa- 

usu- 

almost 

the  capability, 

sionally 

ally 

always 

please  leave  that 

(less  than 

(half  to 

(over 

row  blank) 

half  the 

80Z  of 

80Z  of 

never 

time) 

time) 

time) 

Can  you  distribute  the  msgs 

as  easily  as  you  need  to? 

Can  you  read/file  msgs  as 

well  as  you  need  to? 

Can  you  retrieve  the  msg 

you  need? 

Can  you  prepare  text /msgs 

as  well  as  you  need  to? 

Can  you  send  notes,  etc. 

as  easily  as  needed? 

Is  the  response  time  to  in- 

structions  satisfactory? 

Do  you  have  problems  with  re- 

liability  (hangs/crashes)?  

Remarks 


ACCEPTABILITY  OF  THE  SYSTEM  FOR  THE  EXPERIMENT 


What  features  or  functions  of  the  MME  may,  in  your  view,  limit  the  usefulness 
of  this  system  during  the  experiment? 


unaccep- 

serious 

table 

not  a 

some  limit  or 

limita- 

limit 

limit  deficiency 

tion 

Remarks 

msg  distribution 
message  retrieval 
msg.  reading/filing 
msg/text  preparation 
msg  coord/release 
response  time 
reliability 
Please  comment 
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MBM 


User  Type  NCOs  J301  NCOs  CCWT  OFFICERS  ACTION  OFFICERS 


Use:  Never  * 0 Perceptions:  Excellent  * E Acceptability:  Excellent  < 

Sometimes*  1 Satisfactory  * S Satisfactory  ' 

Often  " l Poor  * P Poor  1 

Unsatisfactory*  U Unsatisfactory 
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Numerical  codes: 

Use:  never  ■ 0 Perceptions:  excellent  ■ E Acceptability:  excellent  • E 

sometimes  * 1 satisfactory  » S satisfactory  • S 

often  ■ 2 poor  ■ P poor  ■ P 

unsatisfactory*  U unsatisfactory*  U 
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The  following  charts  characterize  the  use  made  of  SIGMA.  The  SIGMA 


instructions  have  been  grouped  by  type  (e.g.,  display  message,  display  file). 
Each  graph  represents  the  use  made  by  a user  acting  in  one  role  over  a period 
of  time;  there  are  four  time  periods  covered  by  the  report.  The  broken  lines 
r*Pr«*«nt  the  number  of  instructions  and  the  solid  lines  represent  the  machine 
processing  time.  The  scale  is  in  percentage.  For  example,  J32  during  the  6 
Sept  - 27  Sept  period,  executed  a total  of  114  display  message-type  coonands; 
these  constituted  approximately  15Z  of  the  commands  he  issued  and  constituted 
approximately  40Z  of  the  machine  processing  time  he  used  during  the  period. 

On  each  page,  the  commands  are  grouped  into  two  general  categories:  displaying 
or  routing  messages  and  creating  messages. 


PERCENT 


COPY  MESSAGE  t-  SAVE/UPDATE 


SURFACE 

6 September  - 27  September 


FIND  ENTRY 


J32 


FIND  ENTRY 


SURFACE 

6 September  - 27  September 


JRC 

6 September  - 27  September 


DISPLAY  HESSACE  U 13  CREATE  MESSACE 


FIND  ENTRY 


September  - 27-  September 


PERCENT  PERCENT 


17  August  - 30  August 


FIND  ENTRY  |-  I OTHERS 


■ 30  August 


II®  «*IA  I -I  9N1US  QNIi 


17  August  - 30  August 


DISPLAY  MESSAGE  1111111  ra  | CREATE  MESSAGE 


PERCENT 


J34 

17  August  - 30  August 


H Ilium  92  I CREATE  MESSAGE 


1 


FIND  ENTM  h OTHERS 


J311 

17  August  - 30  August 


FIND  STXINC  VIEW  DIRECTORY 


DISPLAY  MESSAGE  I-  CREATE  MESSAGE 


PERCENT 


Clerks 

17  August  - 30  August 


3 August  - 16  August 


FIND  ENTRY  h I OTHERS 


.1 

f 


J342 

3 Aufuat  - 16  August 


DISPLAY  MESSAGE  JHHH  »l  Mill,  CREATE  MESSAGE 


PERCENT 


SURFACE 

3 August  - 16  August 


PERCENT 


J301 

3 Aufejllt  - 16  Augunt 


DISPLAY  MESSAGE  li  2 CREATE  MESSAGE 


OTHERS 


Clerks 

3 August  - 16  August 


DISPLAY  MESSAGE  l 1 CHEAT!  MESSAGE 


19  July  - 2 Auguat 


SURFACE 

19  July  - 2 August 


DISPLAY  MESSAGE  Uiitntmim  . 67  I CREATE  MESSAGE 


PERCENT 


2 August 


VIEW  Dll 


PERCENT 


Clerk* 


FIND  STRINC  f-  VIEW  DIRECTORY 


19  .Inly  - 2 August 


ON  IMIS  OKU 


